Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-129847: Add graphlib.reverse(), graphlib.as_transitive() #130875

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
73 changes: 70 additions & 3 deletions Doc/library/graphlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@

--------------

Topological Ordering
--------------------

.. class:: TopologicalSorter(graph=None)

Expand Down Expand Up @@ -194,15 +196,80 @@
.. versionadded:: 3.9


Graph Functions
---------------

Some functions are provided to work with directed acyclic graph structures using
the same representation accepted by :class:`.TopologicalSorter`: a mapping of
nodes to iterables of predecessors.

Nodes can be any :term:`hashable` object. Duplicate edges are ignored.

For example, it can be useful to construct a TopologicalSorter that processes
a graph in reverse order. This can be done by passing the :func:`reverse` of a
graph to the constructor of :class:`TopologicalSorter`:

.. code-block:: python

ts = TopologicalSorter(reverse(graph))


.. function:: reverse(graph)

Return a new graph with the edges reversed.

The *graph* argument must be a dictionary representing a directed graph
where the keys are nodes and the values are iterables of predecessors. Keys
and predecessors must be hashable.

Return a dict mapping nodes to sets of successors. The returned graph will
include a key for all nodes in the input graph, possibly with an empty set
of successors:

.. code-block:: pycon

>>> reverse({"a": ["b", "c"], "d": [], "c": ["e"]})
{'b': {'a'}, 'c': {'a'}, 'a': set(), 'd': set(), 'e': {'c'}}

.. versionadded:: next


.. function:: as_transitive(graph)

Compute the transitive closure of a dependency graph.

The *graph* argument must be a dictionary representing a directed graph
where the keys are nodes and the values are iterables of predecessors. Keys
and predecessors must be hashable.

If A is a direct predecessor of B, and B is a direct predecessor of C,
then A is a transitive predecessor of C. The returned dict maps each key in
the input graph to a sets of all such transitive predecessors for that key.

If the input graph contains cycles, raise a :exc:`CycleError`.

Nodes that do not appear as keys in the input graph, but appear as
predecessors of other nodes, will not be included as keys in the returned
transitive graph:

.. code-block:: pycon

>>> as_transitive({"a": ["b"], "b": ["c"]})
{'a': {'b', 'c'}, 'b': {'c'}}

.. versionadded:: next


Exceptions
----------
The :mod:`graphlib` module defines the following exception classes:

.. exception:: CycleError

Subclass of :exc:`ValueError` raised by :meth:`TopologicalSorter.prepare` if cycles exist
in the working graph. If multiple cycles exist, only one undefined choice among them will
be reported and included in the exception.
Subclass of :exc:`ValueError` raised by :meth:`TopologicalSorter.prepare` and
:func:`as_transitive` when a cycle is detected in the graph. If multiple
cycles exist, only one undefined choice among them will be reported and
included in the exception.

The detected cycle can be accessed via the second element in the :attr:`~BaseException.args`
attribute of the exception instance and consists in a list of nodes, such that each node is,
Expand Down
9 changes: 9 additions & 0 deletions Doc/whatsnew/3.14.rst
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,15 @@ getopt
* Add support for returning intermixed options and non-option arguments in order.
(Contributed by Serhiy Storchaka in :gh:`126390`.)


graphlib
--------
Comment on lines +588 to +589
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be re-targeted to 3.15.


* Add :func:`graphlib.reverse` and :func:`graphlib.as_transitive` to work with
directed acyclic graphs represented as dicts.
(Contributed by Daniel Pope in :gh:`129847`.)


http
----

Expand Down
85 changes: 84 additions & 1 deletion Lib/graphlib.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from types import GenericAlias

__all__ = ["TopologicalSorter", "CycleError"]
__all__ = ["TopologicalSorter", "CycleError", "reverse", "as_transitive"]

_NODE_OUT = -1
_NODE_DONE = -2
Expand Down Expand Up @@ -248,3 +248,86 @@ def static_order(self):
self.done(*node_group)

__class_getitem__ = classmethod(GenericAlias)


def reverse(graph):
"""Reverse the direction of the edges in a directed graph.

Given a mapping from nodes to collections of their dependencies,
construct a dict mapping each node to the set of nodes that depend on it.

Nodes that have no dependents appear as keys in the result, with an empty
set as value.

For example:

>>> reverse({"a": ["b", "c"], "d": []})
{'b': {'a'}, 'c': {'a'}, 'a': set(), 'd': set()}

"""
result = {}
for node, deps in graph.items():
for dep in deps:
if dep not in result:
result[dep] = {node}
else:
result[dep].add(node)
if node not in result:
result[node] = set()
return result


def as_transitive(graph):
"""Compute the transitive closure of a dependency graph.

If the input graph contains cycles, raise a CycleError.

The returned dict will contain the same keys as the input graph, but the
values will be sets of transitive predecessors of the key, rather than
direct predecessors.

For example:
>>> as_transitive({"a": ["b"], "b": ["c"]})
{'a': {'b', 'c'}, 'b': {'c'}}
"""
unprocessed = dict(graph)
transitive_graph = {node: set() for node in graph}

while unprocessed:
node, deps = unprocessed.popitem()

# We use a recursive algorithm but don't use Python's stack in
# order to avoid hitting the recursion limit.
stack = [iter(deps)]
path = [node] # Ordering for cycle detection
seen = {node} # Fast test for cycle detection

while stack:
try:
child = next(stack[-1])
except StopIteration:
# "Return"
stack.pop()
if not stack:
break
child = path.pop()
seen.remove(child)
node = path[-1]
transitive_graph[node].update(transitive_graph.get(child, ()))

transitive_graph[node].add(child)

if child in seen:
cycle = [child, *reversed(path[path.index(child):])]
raise CycleError("nodes are in a cycle", cycle)

# "Recurse" into the child's deps if it has any
if deps := unprocessed.pop(child, None):
node = child
stack.append(iter(deps))
path.append(child)
seen.add(child)
continue
transitive_graph[node].update(transitive_graph.get(child, ()))

return transitive_graph
130 changes: 130 additions & 0 deletions Lib/test/test_graphlib.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
import graphlib
import sys
import os
Comment on lines +2 to 3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: sorting

Suggested change
import sys
import os
import os
import sys

import unittest
from itertools import permutations

from test.support.script_helper import assert_python_ok

Expand Down Expand Up @@ -248,5 +250,133 @@ def check_order_with_hash_seed(seed):
self.assertNotEqual(run2, "")
self.assertEqual(run1, run2)


class TestReverse(unittest.TestCase):
"""Tests for graphlib.reverse()."""

def test_reverse_empty(self):
"""An empty graph has an empty reverse."""
self.assertEqual(graphlib.reverse({}), {})

def test_reverse_simple(self):
"""We can reverse a simple graph."""
graph = {"a": ["b", "c"]}
expected = {"b": {"a"}, "c": {"a"}, "a": set()}
self.assertEqual(graphlib.reverse(graph), expected)

def test_reverse_with_empty_dependencies(self):
"""Nodes with no predecessors are included in the output."""
graph = {"a": []}
expected = {"a": set()}
self.assertEqual(graphlib.reverse(graph), expected)

def test_reverse_with_int_keys(self):
"""Nodes may be any hashable type, such as int."""
graph = {1: {2, 3}, 2: {3}}
expected = {2: {1}, 3: {1, 2}, 1: set()}
self.assertEqual(graphlib.reverse(graph), expected)

def test_reverse_with_iterators(self):
"""Values may be any iterable including iterators."""
graph = {"a": iter("cd"), "b": iter("ace"), "e": iter("f")}
expected = {
"a": {"b"},
"b": set(),
"c": {"a", "b"},
"d": {"a"},
"e": {"b"},
"f": {"e"},
}
self.assertEqual(graphlib.reverse(graph), expected)

class TestAsTransitive(unittest.TestCase):
"""Tests for graphlib.as_transitive()."""
Comment on lines +292 to +293
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with the recursive version is that it is limited to graphs with diameter < sys.getrecursionlimit() so 1000 by default.

Could we add a test for such a pathological graph?


def test_as_transitive_empty(self):
"""An empty graph has an empty transitive closure."""
self.assertEqual(graphlib.as_transitive({}), {})

def test_as_transitive_no_dependencies(self):
"""Nodes with no predecessors are included in the output."""
graph = {"a": [], "b": []}
expected = {"a": set(), "b": set()}
self.assertEqual(graphlib.as_transitive(graph), expected)

def test_as_transitive_simple(self):
"""We can compute the transitive closure of a simple graph.

Given the input does not include "d" as a key, the output omits it.
"""
graph = {"a": ["b"], "b": ["c", "e"], "c": ["d"]}
expected = {"a": {"b", "c", "d", "e"}, "b": {"c", "d", "e"}, "c": {"d"}}
self.assertEqual(graphlib.as_transitive(graph), expected)

def test_as_transitive_disordered_chain(self):
"""We can compute the transitive closure of a graph in any order."""
graph = {"a": "b", "b": "c", "c": "d", "d": "e"}.items()
expected = {
"a": {"b", "c", "d", "e"},
"b": {"c", "d", "e"},
"c": {"d", "e"},
"d": {"e"}
}
for perm in permutations(graph):
with self.subTest(perm):
self.assertEqual(graphlib.as_transitive(dict(perm)), expected)

def test_as_transitive_disordered_wide(self):
"""We can compute the transitive closure of a graph in any order."""
graph = {"a": "bc", "c": "de", "e": "f"}.items()
expected = {
"a": {"b", "c", "d", "e", "f"},
"c": {"d", "e", "f"},
"e": {"f"}
}
for perm in permutations(graph):
with self.subTest(perm):
self.assertEqual(graphlib.as_transitive(dict(perm)), expected)

def test_large_diameter_graph(self):
"""We can compute the transitive closure of a large graph."""
size = sys.getrecursionlimit()
graph = {k: [k + 1] for k in range(size)}
expected = {k: set(range(k + 1, size + 1)) for k in range(size)}
self.assertEqual(graphlib.as_transitive(graph), expected)

def test_as_transitive_disjoint(self):
"""We compute the transitive closure of disjoint subgraphs."""
graph = {"a": "b", "b": "c", 1: [2], 2: [3]}
expected = {"a": {"b", "c"}, "b": {"c"}, 1: {2, 3}, 2: {3}}
self.assertEqual(graphlib.as_transitive(graph), expected)

def test_as_transitive_with_tuple_keys(self):
"""Nodes may be any hashable type, such as tuple."""
graph = {('a',): [('b',)], ('b',): [('c',)]}
expected = {('a',): {('b',), ('c',)}, ('b',): {('c',)}}
self.assertEqual(graphlib.as_transitive(graph), expected)

def test_as_transitive_with_int_keys_and_list_values(self):
"""Nodes may be any hashable type, such as int."""
graph = {1: [2], 2: [3], 3: []}
expected = {1: {2, 3}, 2: {3}, 3: set()}
self.assertEqual(graphlib.as_transitive(graph), expected)

def test_as_transitive_with_iterators(self):
"""Values may be any iterable including iterators."""
graph = {"a": iter("cd"), "b": iter("ace"), "e": iter("f")}
expected = {"a": {"c", "d"}, "b": {"c", "d", "e", "a", "f"}, "e": {"f"}}
self.assertEqual(graphlib.as_transitive(graph), expected)

def test_as_transitive_cyclic(self):
"""Raise CycleError if a cycle is detected."""
graph = {"a": ["b"], "b": ["c"], "c": ["a"]}
with self.assertRaises(graphlib.CycleError) as cm:
graphlib.as_transitive(graph)
self.assertEqual(
cm.exception.args,
("nodes are in a cycle", ["c", "b", "a", "c"]),
)


if __name__ == "__main__":
unittest.main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Add :func:`graphlib.reverse` and :func:`graphlib.as_transitive` to work with
directed acyclic graphs represented as dicts.
Patch by Daniel Pope.
Loading