support git 2.19's commit-graph #5757

lhchavez · 2021-01-04T01:09:49Z

Starting from version 2.19, git supports commit-graph, which allows for some significant improvements in the performance of several graph-based operations. Notably computing the merge base between two (or more) commits and determining whether one commit is ahead/behind of another.

This should be relatively straightforward to implement. Similar to #5399, this can be split into several chunks for ease of reviewing:

Support for parsing existing commit-graph files, and a fuzzer because parsing is hard. commit-graph: Introduce a parser for commit-graph files #5762
Support for reading the commit metadata and generation numbers from existing commit-graph files. (in other words, this will ignore any path-based bloom filters). commit-graph: Support lookups of entries in a commit-graph #5763
Support knowing whether a commit-graph file has been modified since it was parsed commit-graph: Introduce git_commit_graph_needs_refresh() #5764
Modify git_commit_list_parse() to take advantage of the commit-graph metadata, if available, which avoids going to the odb, looking up the object, and parsing the commit. commit-graph: Use the commit-graph in revwalks #5765
Modify the functions mark_parents() and paint_down_to_common() to take advantage of the commit-graph metadata, if available, which can prevent multiple visitations due to commit time inversions. commit-graph: Introduce git_commit_list_generation_cmp #5766
Refactor git_merge__bases_many() so that it can take a minimum generation number. That way git_graph_descendant_of() can specify this minimum generation number to cut down the BFS earlier than what it would in a regular git_merge_base(), and implement git_graph_reachable_from_any() which is analogous to git_merge_base_many() and uses the fancy new git_merge__bases_many() that knows about the minimum generation number. This can be used to have the concept of private/protected/public branches, so that users of a repository have different permissions depending on the reachability of a commit from any of a set of tips. (a.k.a. the whole reason I am writing all of this, ha) graph: Create git_graph_reachable_from_any() #5767
Support for creating a commit-graph file from an open repository. commit-graph: Add a way to write commit-graph files #5778
Support for creating commit-graph chains, so that the cost of updating a commit-graph is reduced to only the commits that have changed since the last file / chain was written.

Relevant documentation:

The text was updated successfully, but these errors were encountered:

This change is the first in a series to add support for git's commit-graph. This should speed up commit graph traversals by avoiding object parsing and allowing some operations to terminate earlier. Part of: libgit2#5757

This change introduces a new API function `git_graph_reachable_from_any()`, that answers the question whether a commit is reachable from any of the provided commits through following parent edges. This function can take advantage of optimizations provided by the existence of a `commit-graph` file, since it makes it faster to know whether, given two commits X and Y, X cannot possibly be an reachable from Y. Part of: libgit2#5757

This change is the first in a series to add support for git's commit-graph. This should speed up commit graph traversals by avoiding object parsing and allowing some operations to terminate earlier. Part of: libgit2#5757

This change introduces `git_commit_graph_entry_find()` and `git_commit_graph_entry_parent()`. These two functions allow a much faster lookup of commits by ID, since the ODB does not need to be consulted, the commit object does not need to be inflated, and the contents of the commit object do not need to be parsed. Part of: libgit2#5757

This change introduces a function that allows the caller to know whether the `commit-graph` file has not been modified since it was parsed. Part of: libgit2#5757

This change makes revwalks a bit faster by using the `commit-graph` file (if present). This is thanks to the `commit-graph` allow much faster parsing of the commit information by requiring near-zero I/O (aside from reading a few dozen bytes off of a `mmap(2)`-ed file) for each commit, instead of having to read the ODB, inflate the commit, and parse it. Part of: libgit2#5757

This change makes calculations of merge-bases a bit faster when there are complex graphs and the commit times cause visiting nodes multiple times. This is done by visiting the nodes in the graph in reverse generation order when the generation number is available instead of commit timestamp. If the generation number is missing in any pair of commits, it can safely fall back to the old heuristic with no negative side-effects. Part of: libgit2#5757

This change introduces a new API function `git_graph_reachable_from_any()`, that answers the question whether a commit is reachable from any of the provided commits through following parent edges. This function can take advantage of optimizations provided by the existence of a `commit-graph` file, since it makes it faster to know whether, given two commits X and Y, X cannot possibly be an reachable from Y. Part of: libgit2#5757

This change is the first in a series to add support for git's commit-graph. This should speed up commit graph traversals by avoiding object parsing and allowing some operations to terminate earlier. Part of: libgit2#5757

This change introduces `git_commit_graph_entry_find()` and `git_commit_graph_entry_parent()`. These two functions allow a much faster lookup of commits by ID, since the ODB does not need to be consulted, the commit object does not need to be inflated, and the contents of the commit object do not need to be parsed. Part of: libgit2#5757

This change introduces a function that allows the caller to know whether the `commit-graph` file has not been modified since it was parsed. Part of: libgit2#5757

This change makes revwalks a bit faster by using the `commit-graph` file (if present). This is thanks to the `commit-graph` allow much faster parsing of the commit information by requiring near-zero I/O (aside from reading a few dozen bytes off of a `mmap(2)`-ed file) for each commit, instead of having to read the ODB, inflate the commit, and parse it. This is done by modifying `git_commit_list_parse()` and letting it use the ODB-owned commit-graph file. Part of: libgit2#5757

This change makes calculations of merge-bases a bit faster when there are complex graphs and the commit times cause visiting nodes multiple times. This is done by visiting the nodes in the graph in reverse generation order when the generation number is available instead of commit timestamp. If the generation number is missing in any pair of commits, it can safely fall back to the old heuristic with no negative side-effects. Part of: libgit2#5757

This change introduces a new API function `git_graph_reachable_from_any()`, that answers the question whether a commit is reachable from any of the provided commits through following parent edges. This function can take advantage of optimizations provided by the existence of a `commit-graph` file, since it makes it faster to know whether, given two commits X and Y, X cannot possibly be an reachable from Y. Part of: libgit2#5757

This change adds the git_commit_graph_writer_* functions to allow to write and create `commit-graph` files from `.idx`/`.pack` files or `git_revwalk`s. Part of: libgit2#5757

This change is the first in a series to add support for git's commit-graph. This should speed up commit graph traversals by avoiding object parsing and allowing some operations to terminate earlier. Part of: #5757

This change introduces `git_commit_graph_entry_find()` and `git_commit_graph_entry_parent()`. These two functions allow a much faster lookup of commits by ID, since the ODB does not need to be consulted, the commit object does not need to be inflated, and the contents of the commit object do not need to be parsed. Part of: #5757

This change introduces a function that allows the caller to know whether the `commit-graph` file has not been modified since it was parsed. Part of: #5757