From c98d0bc317e85cc56765d8a33d922905c3773c5b Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Thu, 24 Dec 2020 08:47:01 -0700 Subject: [PATCH 01/44] make the process more front and center in MFC.md --- MFC.md | 139 ++++++++++++++++++++++++++++++++++++--------------------- 1 file changed, 89 insertions(+), 50 deletions(-) diff --git a/MFC.md b/MFC.md index f534985..657b435 100644 --- a/MFC.md +++ b/MFC.md @@ -5,7 +5,95 @@ Note: This document uses the convention where the upstream origin name is `freebsd` as suggested in other docs. -## Goals +## Summary + +MFC workflow can be summarized as `git cherry-pick -x` plus git commit +--amend to adjust the commit message. For multiple commits, use `git rebase -i` +to squash them together and edit the commit message. + +## Single commit MFC + +``` +% git checkout stable/X +% git cherry-pick -x $HASH --edit +``` + +For MFC commits, for example a vendor import, you would need to specify one parent for cherry-pick +purposes. Normally, that would be the "first parent" of the branch you are cherry-picking from, so: + +``` +% git checkout stable/X +% git cherry-pick -x $HASH -m 1 --edit +``` + +If things go wrong, you'll either need to abort the cherry-pick with `git cherry-pick --abort` or fix it +up and do a `git cherry-pick --continue`. + +Once the cherry-pick is finished, push with `git push`. If you get an error due to losing the commit race, +use `git pull --rebase` and try to push again. + +## Multiple commit MFC + +``` +% git checkout -b tmp-branch stable/X +% for h in $HASH_LIST; do git cherry-pick -x $h; done +% git rebase -i stable/X +# mark each of the commits after the first as 'squash' +# edit the commit message to be sane +% git push freebsd HEAD:stable/X +``` + +If the push fails due to losing the commit race, rebase and try again: + +``` +% git checkout stable/X +% git pull +% git checkout tmp-branch +% git rebase stable/X +% git push freebsd HEAD:stable/X +``` + +Once the MFC is complete, you can delete the temporary branch: + +``` +% git checkout stable/X +% git branch -d tmp-branch +``` + +## MFC a vendor import + +Vendor imports are the only thing in the tree that creates a merge +commit in the main line. Cherry picking merge commits into stable/XX +presents an additional difficulty because there are two parents for a +merge commit. Generally, you'll want the first parent's diff since +that's the diff to mainline (though there may be some exceptions). + +``` +% git cherry-pick -x -m 1 $HASH +``` +is typically what you want. This will tell cherry-pick to apply the correct diff. + +There are some, hopefully, rare cases where it's possible that the +mainline was merged backwards by the conversion script. Should that be +the case (and we've not found any yet), you'd change the above to '-m 2' +to pickup the proper parent. Just do +``` +% git cherry-pick --abort +% git cherry-pick -x -m 2 $HASH +``` +to do that. The `--aboort` will cleanup the failed first attempt. + +## Redoing a MFC + +If you do a MFC, and it goes horribly wrong and you want to start over, +then the easiest way is to use `git reset --hard` like so: +``` +% git reset --hard freebsd/stable/12 +``` +though if you have some revs you want to keep, and others you don't, +using 'git rebase -i' is better. + +## Considerations when MFCing When committing source commits to stable and releng branches, we have the following goals: @@ -145,55 +233,6 @@ commit message. The bottom line is that developers will likely need to curate their commit message for MFCs that are non-trivial. -## Single commit MFC - -``` -% git checkout stable/X -% git cherry-pick -x $HASH --edit -``` - -For MFC commits, for example a vendor import, you would need to specify one parent for cherry-pick -purposes. Normally, that would be the "first parent" of the branch you are cherry-picking from, so: - -``` -% git checkout stable/X -% git cherry-pick -x $HASH -m 1 --edit -``` - -If things go wrong, you'll either need to abort the cherry-pick with `git cherry-pick --abort` or fix it -up and do a `git cherry-pick --continue`. - -Once the cherry-pick is finished, push with `git push`. If you get an error due to losing the commit race, -use `git pull --rebase` and try to push again. - -## Multiple commit MFC - -``` -% git checkout -b tmp-branch stable/X -% for h in $HASH_LIST; do git cherry-pick -x $h; done -% git rebase -i stable/X -# mark each of the commits after the first as 'squash' -# edit the commit message to be sane -% git push freebsd HEAD:stable/X -``` - -If the push fails due to losing the commit race, rebase and try again: - -``` -% git checkout stable/X -% git pull -% git checkout tmp-branch -% git rebase stable/X -% git push freebsd HEAD:stable/X -``` - -Once the MFC is complete, you can delete the temporary branch: - -``` -% git checkout stable/X -% git branch -d tmp-branch -``` - ## Looking for things to MFC If you are looking for changes to MFC, the following may help: From 51c9f03ea82d5e5c7f62bfcb4a960df4f8779536 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Thu, 24 Dec 2020 08:54:55 -0700 Subject: [PATCH 02/44] Update faq.md Add pointer to thj's writeup --- faq.md | 1 + 1 file changed, 1 insertion(+) diff --git a/faq.md b/faq.md index 10a5360..43f825d 100644 --- a/faq.md +++ b/faq.md @@ -50,6 +50,7 @@ accidentally getting into a 'merge nightmare' where you have an extra change in your tree, forcing a complicated merge rather than a simple one. +Here's https://adventurist.me/posts/00296 a good writeup that goes into more detail. ## Developers ### Ooops! I committed to `main` instead of a branch. From a0106911f010ada9cc8f2d2496b5297505ea7d2f Mon Sep 17 00:00:00 2001 From: Guangyuan Yang Date: Thu, 24 Dec 2020 15:04:47 -0500 Subject: [PATCH 03/44] Add SUMMARY.md for a clean doc structure --- README.md | 12 +++++++++--- SUMMARY.md | 43 ++++++++++++++++++++++++++++++++++++++++++ TODO.md => doc-todo.md | 0 3 files changed, 52 insertions(+), 3 deletions(-) create mode 100644 SUMMARY.md rename TODO.md => doc-todo.md (100%) diff --git a/README.md b/README.md index 5bddcbf..23859b9 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,17 @@ This repo contains the draft docs for the FreeBSD migration to Git. -The best place to start is with the mini-primer.md if you are coming to this fresh. Otherwise the best place to start is src-cvt.md if you have an existing source tree from the project you'd like to convert over to the new git repo. Once you've read that, you can read mini-primer.md for a general primer. +## How to read the docs -More advanced topics can be found in the FAQ.md. +Please check out the [Summary](SUMMARY.md) page for a clean and complete list of the documentation. -The other files contain inforamtion about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. +The best place to start is with the [FreeBSD mini-git Primer](mini-primer.md) if you are coming to this fresh. Otherwise the best place to start is the [FreeBSD Src Committer Transition Guide](src-cvt.md) if you have an existing source tree from the project you'd like to convert over to the new git repo. Once you've read that, you can read the [FreeBSD mini-git Primer](mini-primer.md) for a general primer. + +More advanced topics can be found in the [Git FAQ](FAQ.md). + +Other files contain inforamtion about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. + +## Contributing Please feel free to submit pull requests for typos, clearer language, additional topics, etc. diff --git a/SUMMARY.md b/SUMMARY.md new file mode 100644 index 0000000..e184722 --- /dev/null +++ b/SUMMARY.md @@ -0,0 +1,43 @@ +# Summary + +## Preface + +* [README](README.md) +* [FreeBSD moving to Git: Why](git-why.md) + +## Getting Started + +* [How FreeBSD implements Git](git-how.md) +* [FreeBSD mini-git Primer](mini-primer.md) +* [Important URLs of the FreeBSD Git Infrastructure](URLs.md) + +## `doc` Specific + +* [FreeBSD Doc Committer Transition Guide](doc-cvt.md) +* [~~FreeBSD Doc tree migration schedule~~](doc-cutover-schedule.md) (Migration Completed as of Dec 9, 2020) +* [Todo List for -doc](doc-todo.md) + +## `src` Specific + +* [FreeBSD Src Committer Transition Guide](src-cvt.md) +* [(WIP) How to do vendor imports](vendor.md) +* [(WIP) How to MFC](MFC.md) +* [A brief note about Subversion Projects and User branches](projects-user.md) + +## Git References + +* [Git FAQ](faq.md) + +## Committer References + +* [Writing Good FreeBSD Commit Messages](commit.md) +* [Include appropriate metadata in a footer](meta.md) + +## Design & Implementation + +* [Big Picture](big-picture.md) +* [Design Notes](design-notes.md) + +## Next Steps + +* [Github mirroring plan](github-mirroring.md) diff --git a/TODO.md b/doc-todo.md similarity index 100% rename from TODO.md rename to doc-todo.md From 45a524b6d95e3a723b8bbf7ea9d46e5aa3d40cf2 Mon Sep 17 00:00:00 2001 From: "Robert N. M. Watson" Date: Thu, 24 Dec 2020 09:06:51 +0000 Subject: [PATCH 04/44] Explain the two repositories differently. --- mini-primer.md | 30 ++++++++++++++++-------------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/mini-primer.md b/mini-primer.md index 1cad4d2..0cc330e 100644 --- a/mini-primer.md +++ b/mini-primer.md @@ -32,20 +32,22 @@ is the default branch if you omit the '-b branch' or '--branch branch' options below. ### Repositories -At the moment, there are two repositories. The hashes are different -between them. The old GitHub repo is similar to the new cgit -repo. However, there are a large number of mistakes in the GitHub repo -that required us to regenerate the export when we migrated to having a -git repo be the source of truth for the project. - -The GitHub repo is at https://github.com/freebsd/freebsd.git The new -cgit beta repo is at https://cgit-beta.freebsd.org/src.git and the -production URLs will be https://git.freebsd.org/src.git and -ssh://anongit@git.FreeBSD.org/src.git (or anongit@git.FreeBSD.org:src.git) -after the cut over. These will -be $URL in the commands below. Please note that the cgit repo is -still beta at this time, and hashes may change as its refined. Once we -move into production, no further hash changes will happen. +At the moment, there are two repositories: + +- The old GitHub repository (https://github.com/freebsd/freebsd.git) + was exported by the project for several years to support downstream + users using git. There were a large number of mistakes in the repo + that have led us to generate a new export to be the source of truth, + which will as a result have new hashes. + +- The new cgit beta repo (https://cgit-beta.freebsd.org/src.git) is in + the process of becoming the authoritative repository (and history) + for the project. It is currently being refined, which may lead to + further hash changes. Once it moves into production (and the hashes + are finalized), its URls will be https://git.freebsd.org/src.git, + https://git.freebsd.org/src.git (or anongit@git.FreeBSD.org:src.git). + +The new repository should be used as $URL in the commands below. Note: The project doesn't use submodules as they are a poor fit for our workflows and development model. How we track changes in From 4c43895cb34260253577fb9a3534cb9f02040dd3 Mon Sep 17 00:00:00 2001 From: rpokala-freebsd <76583413+rpokala-freebsd@users.noreply.github.com> Date: Fri, 25 Dec 2020 00:56:39 -0800 Subject: [PATCH 05/44] Fix initial `git clone` command to match URLs.md The initial `git clone` command here doesn't match the one from URLs.md. I suspect that one is correct, for the simple reason that it's explicitly picking up more stuff. --- faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/faq.md b/faq.md index 43f825d..86d6562 100644 --- a/faq.md +++ b/faq.md @@ -26,7 +26,7 @@ First, you need to clone the FreeBSD repository, shown here cloning into `freebsd-current` to reduce confusion. $URL is whatever mirror works best for you: ``` -% git clone -o freebsd $URL freebsd-current +% git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' $URL freebsd-current ``` then once that's cloned, you can simply create a worktree from it: ``` From 87028b0ad454969a6f83f03c0e653659900b52d7 Mon Sep 17 00:00:00 2001 From: Guangyuan Yang Date: Fri, 25 Dec 2020 13:25:47 -0500 Subject: [PATCH 06/44] Fix a typo in README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 23859b9..d234dc0 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ The best place to start is with the [FreeBSD mini-git Primer](mini-primer.md) if More advanced topics can be found in the [Git FAQ](FAQ.md). -Other files contain inforamtion about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. +Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. ## Contributing From 39e69a7944f39a012adbcd7f678924a0b3bd3e39 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ulrich=20Sp=C3=B6rlein?= Date: Sun, 27 Dec 2020 15:18:39 +0100 Subject: [PATCH 07/44] Drop the "beta" in some places and make legacy vs. new stand out more. Also hint at `git reflog`, which is useful in recovering various "catastrophes" --- mini-primer.md | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/mini-primer.md b/mini-primer.md index 0cc330e..2ad20da 100644 --- a/mini-primer.md +++ b/mini-primer.md @@ -40,12 +40,11 @@ At the moment, there are two repositories: that have led us to generate a new export to be the source of truth, which will as a result have new hashes. -- The new cgit beta repo (https://cgit-beta.freebsd.org/src.git) is in - the process of becoming the authoritative repository (and history) - for the project. It is currently being refined, which may lead to - further hash changes. Once it moves into production (and the hashes - are finalized), its URls will be https://git.freebsd.org/src.git, - https://git.freebsd.org/src.git (or anongit@git.FreeBSD.org:src.git). +- The new git repo (https://cgit.freebsd.org/src.git) is + the authoritative repository (and history) + for the project. Its cloning URLs are https://git.freebsd.org/src.git, + ssh://anongit@git.FreeBSD.org/src.git or + ssh://git@gitrepo.FreeBSD.org/src.git (developers with SSH key only). The new repository should be used as $URL in the commands below. @@ -224,18 +223,18 @@ a good resource for when things go wrong or for unusual cases. The ports tree operates the same way. The branch names are different and the repos are in different locations. -The GitHub mirror is at https://github.com/freebsd/freebsd-ports.git . -The cgit mirror is https://cgit-beta.freebsd.org/ports.git . The +The legacy GitHub mirror is at https://github.com/freebsd/freebsd-ports.git . +The canonical cgit mirror is https://cgit-beta.freebsd.org/ports.git . The production git repo will be https://git.freebsd.org/ports.git and ssh://anongit@git.FreeBSD.org/ports.git (or anongit@git.FreeBSD.org:ports.git) when the time comes. -As with ports, the 'current' branches are 'master' and 'main' +As with ports, the 'current' branches are 'master' (legacy) and 'main' (new) respectively. The quarterly branches are named the same as in FreeBSD's svn repo. ## Coping with Local Changes -This section addresses tracking local changes.If you have no local +This section addresses tracking local changes. If you have no local changes, you can stop reading now (it's the last section and OK to skip). @@ -381,7 +380,7 @@ save the file. The rebase was interrupted, so you have to complete it: % git rebase --continue ``` -which tells git that ls.c has changed and to continue the rebase +which tells git that ls.c has been fixed and to continue the rebase operation. Since there was a conflict, you'll get kicked into the editor to update the commit message if necessary. It the commit message is still accurate, just exit the editor. @@ -389,6 +388,9 @@ message is still accurate, just exit the editor. If you get stuck during the rebase, don't panic. git rebase --abort will take you back to a clean slate. It's important, though, to start with an unmodified tree. +An aside: The above mentioned 'git reflog' comes in handy here, as it will have +a list of all the (intermediate) commits that you can view or inspect or +cherry-pick. For more on this topic, https://www.freecodecamp.org/news/the-ultimate-guide-to-git-merge-and-git-rebase/ From c4f2714b6dc6ffd49fbef7f81e1f5c8317131ff6 Mon Sep 17 00:00:00 2001 From: Li-Wen Hsu Date: Mon, 28 Dec 2020 10:18:13 +0800 Subject: [PATCH 08/44] Add instruction to install prepare-commit-msg hook This section should be extracted to a standalone chapter of setting up environment for developers. --- URLs.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/URLs.md b/URLs.md index 702610c..2adb015 100644 --- a/URLs.md +++ b/URLs.md @@ -107,6 +107,12 @@ https://github.com/freebsd/git_conv#gimme-the-repo ``` Again, note that `gitrepo.freebsd.org` will be canonicalized to `repo.freebsd.org` in the future. +* Install commit message template hook: + ``` + fetch https://cgit.freebsd.org/src/plain/tools/tools/git/hooks/prepare-commit-msg -o .git/hooks + chmod 755 .git/hooks/prepare-commit-msg + ``` + ### "admin" branch The `access` and `mentors` files are stored in a orphan branch, `internal/admin`, in each repository. From da4b0a5c934b9acd18359fd7620afd0e22d60436 Mon Sep 17 00:00:00 2001 From: Li-Wen Hsu Date: Mon, 21 Dec 2020 16:03:24 +0800 Subject: [PATCH 09/44] Mention how to push to internal/admin branch This is useful for committer mentors. --- URLs.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/URLs.md b/URLs.md index 2adb015..5d83648 100644 --- a/URLs.md +++ b/URLs.md @@ -128,6 +128,16 @@ git checkout -b admin internal/admin For browsing `internal/admin` branch on web: https://cgit.freebsd.org/${repo}/log/?h=internal/admin +For pushing, either specify the full refspec: +``` +git push origin HEAD:refs/internal/admin +``` + +Or setting `push.default` to `upstream` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow: +``` +git config push.default upstream +``` + ## External mirrors Those mirrors are not hosted in FreeBSD.org but still maintained by the project members. From a532bfe9fece05629f63e9714a828240b127bd68 Mon Sep 17 00:00:00 2001 From: John-Mark Gurney Date: Mon, 28 Dec 2020 10:45:46 -0800 Subject: [PATCH 10/44] fix the urls so that they are useful and you don't have to read the examples. --- doc-cvt.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc-cvt.md b/doc-cvt.md index 9f4d59c..3ef3c57 100644 --- a/doc-cvt.md +++ b/doc-cvt.md @@ -9,14 +9,15 @@ better / at all. ## Old vs New URL translation table Before we get started, here's a handy cheat sheet for old to new URLs. +For complete information, please see [URLs.md]. SVN infra -> Git infra map | Item | SVN | Git | | ---------------------------------------- | ------------------------------- | ----------------------------------- | | Web-based repository browser | https://svnweb.freebsd.org | https://cgit.freebsd.org | -| Distributed mirrors for anonymous readonly checkout/clone | https://svn.freebsd.org svn://svn.freebsd.org | https://git.freebsd.org ssh://anongit@git.freebsd.org | -| Read/write Repository for committers (*) | svn+ssh://(svn)repo.freebsd.org | ssh://git@(git)repo.freebsd.org | +| Distributed mirrors for anonymous readonly checkout/clone | https://svn.freebsd.org svn://svn.freebsd.org | https://git.freebsd.org/doc.git ssh://anongit@git.freebsd.org/doc.git | +| Read/write Repository for committers (*) | svn+ssh://(svn)repo.freebsd.org/doc | ssh://git@(git)repo.freebsd.org/doc.git | (*) Before all repositories in SVN have been migrated, the repo.freebsd.org will be pointing to one of: - svnrepo.freebsd.org From 1cb2dc807f629a38639e260692c548689db0e374 Mon Sep 17 00:00:00 2001 From: Roger Pau Monne Date: Wed, 30 Dec 2020 15:35:03 +0100 Subject: [PATCH 11/44] cvt: add a comment about running git gc before adding a remote Cleaning the current repository can speedup the addition of a new remote, since there are less refs to send upstream for comparison. --- doc-cvt.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/doc-cvt.md b/doc-cvt.md index 3ef3c57..52a4e40 100644 --- a/doc-cvt.md +++ b/doc-cvt.md @@ -224,7 +224,9 @@ historical branches... When migrating branches from a github fork from the old github mirror to the official repo, the process is straight forward. This assumes that you have a `origin` upstream pointing to github, adjust if necessary. -This also assumes a clean tree before starting... +This also assumes a clean tree before starting... It's also recommended to +run `git gc` beforehand in order to speed up the process of adding a new +remote. 1. Add the new `freebsd` source of truth: ``` % git remote add freebsd https://git.freebsd.org/doc.git From 8d05982716b80034b6b5bc3d59479d744c449b68 Mon Sep 17 00:00:00 2001 From: Alexander Richardson Date: Tue, 5 Jan 2021 11:09:37 +0000 Subject: [PATCH 12/44] Suggest symlinks for the commit message hook --- URLs.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/URLs.md b/URLs.md index 5d83648..80cec9d 100644 --- a/URLs.md +++ b/URLs.md @@ -108,6 +108,12 @@ https://github.com/freebsd/git_conv#gimme-the-repo Again, note that `gitrepo.freebsd.org` will be canonicalized to `repo.freebsd.org` in the future. * Install commit message template hook: + If you also cloned the src repository, we recommend creating a symlink to the hook so that future + updates will be received: + ``` + ln -s `realpath /tools/tools/git/hooks/prepare-commit-msg` .git/hooks/ + ``` + If not, you can also download the commit message hook: ``` fetch https://cgit.freebsd.org/src/plain/tools/tools/git/hooks/prepare-commit-msg -o .git/hooks chmod 755 .git/hooks/prepare-commit-msg From 0854e80f9a56f02979ba2626b9396bba63fd57f1 Mon Sep 17 00:00:00 2001 From: Alexander Richardson Date: Tue, 5 Jan 2021 11:14:27 +0000 Subject: [PATCH 13/44] Add --track to "Migrating from GitHub fork" Without `--track` this creates a detached HEAD instead of creating a branch `main` that tracks `freebsd/main` --- src-cvt.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src-cvt.md b/src-cvt.md index 3a530bb..82816d0 100644 --- a/src-cvt.md +++ b/src-cvt.md @@ -224,7 +224,7 @@ This also assumes a clean tree before starting... ``` % git remote add freebsd https://git.freebsd.org/src.git % git fetch freebsd -% git checkout freebsd/main +% git checkout --track freebsd/main ``` 2. Rebase all your WIP branches. For each branch FOO, do the following after fetching the `freebsd` sources and creating a local `main` reference with From 7426c39bfe4c752239be15758eeb6d695a9eae99 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Wed, 6 Jan 2021 12:42:47 -0700 Subject: [PATCH 14/44] Revert "Suggest symlinks for the commit message hook" This turns out to be unsafe. This reverts commit 8d05982716b80034b6b5bc3d59479d744c449b68. --- URLs.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/URLs.md b/URLs.md index 80cec9d..5d83648 100644 --- a/URLs.md +++ b/URLs.md @@ -108,12 +108,6 @@ https://github.com/freebsd/git_conv#gimme-the-repo Again, note that `gitrepo.freebsd.org` will be canonicalized to `repo.freebsd.org` in the future. * Install commit message template hook: - If you also cloned the src repository, we recommend creating a symlink to the hook so that future - updates will be received: - ``` - ln -s `realpath /tools/tools/git/hooks/prepare-commit-msg` .git/hooks/ - ``` - If not, you can also download the commit message hook: ``` fetch https://cgit.freebsd.org/src/plain/tools/tools/git/hooks/prepare-commit-msg -o .git/hooks chmod 755 .git/hooks/prepare-commit-msg From 58b5bc192d199bdf42b51a1fcce00b377d751779 Mon Sep 17 00:00:00 2001 From: Mateusz Piotrowski <0mp@FreeBSD.org> Date: Wed, 6 Jan 2021 11:14:30 +0100 Subject: [PATCH 15/44] Push to remote "freebsd" when committing It seems like this is the convention we try to follow in the documentation. --- URLs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/URLs.md b/URLs.md index 5d83648..7a5aa1e 100644 --- a/URLs.md +++ b/URLs.md @@ -130,7 +130,7 @@ https://cgit.freebsd.org/${repo}/log/?h=internal/admin For pushing, either specify the full refspec: ``` -git push origin HEAD:refs/internal/admin +git push freebsd HEAD:refs/internal/admin ``` Or setting `push.default` to `upstream` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow: From 939ebbbcc7837ef271fbf5173cd0c1774f868173 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ulrich=20Sp=C3=B6rlein?= Date: Wed, 6 Jan 2021 16:02:02 +0100 Subject: [PATCH 16/44] How to handle pull requests Add some documentation on what to fetch and pull to get the various Github pull requests into a local tree and polish them to be included into our main history. --- pull-request.md | 149 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 149 insertions(+) create mode 100644 pull-request.md diff --git a/pull-request.md b/pull-request.md new file mode 100644 index 0000000..8aeda0a --- /dev/null +++ b/pull-request.md @@ -0,0 +1,149 @@ +# How to deal with Pull Requests + +Various Git collaboration tools allow random contributors to send us pull +requests (PR, yes, unfortunately the same abbreviation as Problem Report). With +Git, these are comparatively easy to pull down, inspect and test, as well as +rebase onto our `main` branch and push to our source of truth. + +However, we cannot use the integrated 1-click buttons on these tools, as we see +them as read-only mirrors and the pushes need to go directly to FreeBSD's +infrastructure instead. + +Note that we disallow merges (except for vendor merges) so we want a "linear +history" and need to do the equivalent of a "Rebase and merge". + +# GitHub + +As we had the legacy `master` branch on GitHub for quite some time, there are +old PRs that are based on them. If you pull down anything before PR #449, +you'll likely get a commit that's based on `master` instead of `main`. + +Add GitHub as a remote (only showing -src here), add a refspec to pull down +all pull requests, fetch them, and clean up after our "legacy" master +conversion. This is a one-time step and only needed for GitHub. +``` +% git remote add github https://github.com/freebsd/freebsd-src +% git config --add remote.github.fetch "+refs/pull/*:refs/remotes/github/pull/*" +% git fetch github && git gc +``` + +Or fetch a very specific PR, only. +``` +% git fetch github "refs/pull/422/*:refs/remotes/github/pull/422/*" +remote: Enumerating objects: 19, done. +remote: Counting objects: 100% (19/19), done. +remote: Total 23 (delta 19), reused 19 (delta 19), pack-reused 4 +Unpacking objects: 100% (23/23), 6.23 KiB | 102.00 KiB/s, done. +From github.com:freebsd/freebsd-src + * [new ref] refs/pull/422/head -> github/pull/422/head + * [new ref] refs/pull/422/merge -> github/pull/422/merge +% git show-ref | grep /pull/ +63c82ec5c3833b50e45c31c59a04d7f9a827abb8 refs/remotes/github/pull/422/head +1d1d776bd0f036130446f3d331aef3dd131ef8a3 refs/remotes/github/pull/422/merge +``` + +We can ignore the merge head, as we want to pick the (potentially many) commits +off of the head and rebase them onto our `main`. + +# GitLab et al. + +``` +% git remote add gitlab https://gitlab.com/FreeBSD/freebsd-src.git +% git config --add remote.gitlab.fetch "+refs/merge-requests/*:refs/remotes/gitlab/merge-requests/*" +% git fetch gitlab +``` + +Adapt this to wherever you got the pull or merge request from. + +# Common ways to integrate the Pull Request + +We'll take the GitHub example, but GitLab and co. will be just the same. + +Run `git log github/pull/422/head` and you see 3 commits by a contributor, +followed by commits that all come from "foo ". This is an +indication that it's on the legacy `master` branch. + +While you can use `rebase A..B --onto C`, yours truly is often confused by the +ordering of the arguments and a series of cherry-picks is likely easier. So in +this case, let's create a new branch and see if stuff applies cleanly on top of `main`. + +``` +% git checkout -b pull_422 freebsd/main +% git cherry-pick github/pull/422/head~2 +Auto-merging sys/sys/errno.h +Auto-merging sys/kern/imgact_elf.c +[pull_422 df45283b47f9] * Remove redundant code in __elfN(map_insert) * Do more check on the executable file. * Add a new error return constant ESUCCESS to make the return code more clear. + Author: Wuyang Chung + Date: Wed Jan 22 19:05:14 2020 +0800 + 2 files changed, 36 insertions(+), 99 deletions(-) +% git cherry-pick github/pull/422/head~1 +Auto-merging sys/kern/imgact_elf.c +[pull_422 07bb52c328c8] Modify comment. + Author: Wuyang Chung + Date: Wed Jan 22 19:10:33 2020 +0800 + 1 file changed, 1 insertion(+), 1 deletion(-) +% git cherry-pick github/pull/422/head +Auto-merging sys/sys/errno.h +[pull_422 0a27e19be827] Add #ifndef _POSIX_SOURCE around ESUCCESS. + Author: Wuyang Chung + Date: Fri Jan 24 16:31:41 2020 +0800 + 1 file changed, 2 insertions(+) + +``` + +## Massaging the commit history and content + +Inspect the commits with `git log -p --format=fuller`, note that Author is the +contributor and Committer will be you. AuthorDate will also be way in the past, +this is fine. If the Author fields contains egregious swear words or the like, +maybe get them to change this. + +In our case, there's a trailing whitespace introduced by the commit, something +you might want to fix and `git commit --amend` or use `git rebase -i` to squash +things. Especially since there are 2 trivial changes in a commit and an ifdef +wrapper, this is a case of squashing things down into 1 commit. There are +multiple ways to do this. + +``` +% git rebase -i HEAD~10 + +% vim sys/kern/imgact_elf.c +% git commit --amend sys/kern/imgact_elf.c +``` + +You should amend the commit message with a pointer to the review that might've +happened on the website, similar to phabricator. As a simple pull request +number is ambiguous, please use the full URL. + +``` +Pull Request: https://github.com/freebsd/freebsd-src/pull/422 +``` + +## Double check your work + +We now have a single commit (without trailing whitespace), that looks like so: + +``` +% git log -n1 --format=fuller +commit 61ea2b75b055bd7ef6edf0224adefaf457ce4c4a (HEAD -> pull_422) +Author: Wuyang Chung +AuthorDate: 2020-01-22 19:05:14 +0800 +Commit: Ulrich Spörlein +CommitDate: 2021-01-06 14:14:38 +0100 + + * Remove redundant code in __elfN(map_insert) + * Do more check on the executable file. + * Add a new error return constant ESUCCESS to make the return code more clear. + + Also add #ifndef _POSIX_SOURCE around ESUCCESS. + +% git log --graph --oneline +``` + +## Pushing upstream + +If that passes all the build and runtime checks, it can be pushed to FreeBSD, +but since time has passed, we first need to rebase onto `origin/main` again. As +we branched off of `origin/main`, this is a simple `git pull --rebase` away +(pull always implies fetching first). + From 6135f5d3c9fc1f3b673e7ea086f27c6af5281088 Mon Sep 17 00:00:00 2001 From: David Schlachter Date: Thu, 7 Jan 2021 15:23:03 -0500 Subject: [PATCH 17/44] Fix broken link to FAQ --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d234dc0..ece9865 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ Please check out the [Summary](SUMMARY.md) page for a clean and complete list of The best place to start is with the [FreeBSD mini-git Primer](mini-primer.md) if you are coming to this fresh. Otherwise the best place to start is the [FreeBSD Src Committer Transition Guide](src-cvt.md) if you have an existing source tree from the project you'd like to convert over to the new git repo. Once you've read that, you can read the [FreeBSD mini-git Primer](mini-primer.md) for a general primer. -More advanced topics can be found in the [Git FAQ](FAQ.md). +More advanced topics can be found in the [Git FAQ](faq.md). Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. From 7938ef4af098cd4fc3b7c6b3fce9253bc940b6f2 Mon Sep 17 00:00:00 2001 From: Li-Wen Hsu Date: Sat, 9 Jan 2021 12:15:02 +0800 Subject: [PATCH 18/44] Update GitHub mirror status --- doc-cvt.md | 9 +++++---- src-cvt.md | 11 +++++++---- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/doc-cvt.md b/doc-cvt.md index 52a4e40..d5b4bff 100644 --- a/doc-cvt.md +++ b/doc-cvt.md @@ -216,10 +216,11 @@ hash you can use to refer to this commit. ## Migrating from GitHub fork Note: as of this writing, the https://github.com/freebsd/freebsd-doc -repo ends with the last subversion commit. In the near future, we'll -start mirroring the official repo there. We'll likely retain the -`master` branch that's there now and just push to `main` and all the -historical branches... +is mirroring the official `main` branch, along with a `master` branch which +is the leagcy svn2git result. The `master` branch will not be updated anymore. +We'll likely retain the `master` branch for a certain time, but in the future +it will only be kept in the +[freebsd-doc-legacy](https://github.com/freebsd/freebsd-doc-legacy) repository. When migrating branches from a github fork from the old github mirror to the official repo, the process is straight forward. This assumes that diff --git a/src-cvt.md b/src-cvt.md index 82816d0..3075a94 100644 --- a/src-cvt.md +++ b/src-cvt.md @@ -211,10 +211,13 @@ hash you can use to refer to this commit. ## Migrating from GitHub fork Note: as of this writing, the https://github.com/freebsd/freebsd-src -repo ends with the last subversion commit. In the near future, we'll -start mirroring the official repo there. We'll likely retain the -`master` branch that's there now and just push to `main` and all the -historical branches... +is mirroring all official branches, along with a `master` branch which +is the leagcy svn2git result. The `master` branch will not be updated anymore, +and the [last commit](https://github.com/freebsd/freebsd-src/commit/de1aa3dab23c06fec962a14da3e7b4755c5880cf) +contains the instructions of migrating to new `main` branch. +We'll likely retain the `master` branch for a certain time, but in the future +it will only be kept in the +[freebsd-legacy](https://github.com/freebsd/freebsd-legacy) repository. When migrating branches from a github fork from the old github mirror to the official repo, the process is straight forward. This assumes that From ad5b78fce2630c9d69bf330d0480a0b5ee609251 Mon Sep 17 00:00:00 2001 From: Kyle Evans Date: Sat, 9 Jan 2021 12:24:12 -0600 Subject: [PATCH 19/44] Include some advice on patch authorship Start deprecating "submitted by" lines, opting for either having a Git repository to pull from (i.e. correct metadata already) or setting the commit author manually. Signed-off-by: Kyle Evans --- meta.md | 3 ++- pull-request.md | 22 ++++++++++++++++++++++ 2 files changed, 24 insertions(+), 1 deletion(-) diff --git a/meta.md b/meta.md index 394cbeb..502b726 100644 --- a/meta.md +++ b/meta.md @@ -5,7 +5,6 @@ Commit messages may have one or more of number of standard metadata tags in the | Tag | Description | | -------- | -------- | | PR: | FreeBSD problem report (Bugzilla) number | -| Submitted-by: | ID the original author, if not the committer | | Reported-by: | ID of a 3rd party who reported the issue | | Reviewed-by: | Reviewer ID | | Tested-by: | ID of those who have tested the change | @@ -31,3 +30,5 @@ signed-off-by for commits at this time, but don't want to preclude it in the future. In addition, you can replace - with a space in the above tags as a transition aid. + +"Submitted by" may still appear, but is generally discouraged as a legacy tag. See [How to deal with Pull Requests](pull-request.md) for details on attribution. diff --git a/pull-request.md b/pull-request.md index 8aeda0a..2602683 100644 --- a/pull-request.md +++ b/pull-request.md @@ -57,6 +57,8 @@ Adapt this to wherever you got the pull or merge request from. # Common ways to integrate the Pull Request +## Patches in a Git repository + We'll take the GitHub example, but GitLab and co. will be just the same. Run `git log github/pull/422/head` and you see 3 commits by a contributor, @@ -91,6 +93,26 @@ Auto-merging sys/sys/errno.h ``` +## Other patches + +Various means exist to submit patches that do not haved metadata associated with +them, including the patch author, e.g., Bugzilla. Previously a committer would +commit the received patch and typically add the contributor's name and e-mail +address in the message metadata, in the form of a "Submitted by" line. + +Going forward, the procedure is generally the same with exception of the +contributor metadata. Patches should have the author set to the contributor's +information that would have previously appeared in the "Submitted by" tag, and +the "Submitted by" tag should be entirely omitted. + +``` +% git checkout -b unattributed_patch freebsd/main +% fetch "https://.../foo.diff" +% patch < foo.diff +... +% git commit --author="Submitter Name " ... +``` + ## Massaging the commit history and content Inspect the commits with `git log -p --format=fuller`, note that Author is the From 684c9a72903083d7a7ac324aa23e43851d7082f9 Mon Sep 17 00:00:00 2001 From: Kyle Evans Date: Sat, 9 Jan 2021 12:25:26 -0600 Subject: [PATCH 20/44] SUMMARY: hook up the pull request documentation This is generally useful for committers. Signed-off-by: Kyle Evans --- SUMMARY.md | 1 + 1 file changed, 1 insertion(+) diff --git a/SUMMARY.md b/SUMMARY.md index e184722..9d89ef5 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -32,6 +32,7 @@ * [Writing Good FreeBSD Commit Messages](commit.md) * [Include appropriate metadata in a footer](meta.md) +* [How to deal with Pull Requests](pull-request.md) ## Design & Implementation From 91dc376b7875ec163d66c1fb8a7c1002f9a7fa34 Mon Sep 17 00:00:00 2001 From: Li-Wen Hsu Date: Sat, 16 Jan 2021 00:41:02 +0800 Subject: [PATCH 21/44] Grammar fix --- URLs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/URLs.md b/URLs.md index 7a5aa1e..24daffc 100644 --- a/URLs.md +++ b/URLs.md @@ -115,7 +115,7 @@ https://github.com/freebsd/git_conv#gimme-the-repo ### "admin" branch -The `access` and `mentors` files are stored in a orphan branch, `internal/admin`, in each repository. +The `access` and `mentors` files are stored in an orphan branch, `internal/admin`, in each repository. Following example is how to check out the `internal/admin` branch to a local branch named `admin`: From b32a025ed154fa5316ad7493d5c51eac1d9ecd5d Mon Sep 17 00:00:00 2001 From: Ed Maste Date: Thu, 21 Jan 2021 14:41:11 -0500 Subject: [PATCH 22/44] Use space not dash in "Differential Revision" Phabricator does not recognize Differential-Revision. --- meta.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/meta.md b/meta.md index 502b726..68bcde2 100644 --- a/meta.md +++ b/meta.md @@ -16,7 +16,7 @@ Commit messages may have one or more of number of standard metadata tags in the | Relnotes: | Yes/No whether this change should be included in release notes | | Security: | External reference for a security issue, such as a CVE number | | Sponsored-by: | Organization or event that sponsored work on the change | -| Differential-Revision: | Full URL of code review in FreeBSD's Phabricator instance +| Differential Revision: | Full URL of code review in FreeBSD's Phabricator instance | Signed-off-by: | ID certifies compliance with https://developercertificate.org/ | "ID" indicates either a FreeBSD userid, or a name and email @@ -24,10 +24,10 @@ address. Multiple IDs may be presented as a comma-separated list, or by repeating metadata tags on subsequent lines. This represents a change from the prior FreeBSD practice of using -spaces in the metadata tags. However, this standard conforms to the -wider open-source communities use. The project isn't requiring -signed-off-by for commits at this time, but don't want to preclude it -in the future. +spaces in the metadata tags (other than in `Differential Revision`). +However, this standard conforms to the wider open-source communities use. +The project isn't requiring signed-off-by for commits at this time, but +don't want to preclude it in the future. In addition, you can replace - with a space in the above tags as a transition aid. From 64877319b338380e0dbb0ded26aa1e4a42d50d9a Mon Sep 17 00:00:00 2001 From: Ben Woods Date: Sat, 23 Jan 2021 16:29:40 +0800 Subject: [PATCH 23/44] Exclude .git directory from rsync This prevents the rsync command from clobbering .git --- vendor.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/vendor.md b/vendor.md index 89bdc7a..5beb0be 100644 --- a/vendor.md +++ b/vendor.md @@ -40,7 +40,7 @@ have added or removed files, so we want to make sure deletions are propagated as well. rsync(1) is commonly installed, so I'll use that. ``` % cd ../mtree -% rsync -va --del ~/git/NetBSD/usr.sbin/mtree/ . +% rsync -va --del --exclude=".git" ~/git/NetBSD/usr.sbin/mtree/ . % git add -A % git status ... From 698e802f54a87070ec888a62f29abc47d77e7dd6 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Tue, 2 Feb 2021 11:45:42 -0700 Subject: [PATCH 24/44] Convert readme to asciidoc --- README.ad | 26 ++++++++++++++++++++++++++ README.md | 21 --------------------- 2 files changed, 26 insertions(+), 21 deletions(-) create mode 100644 README.ad delete mode 100644 README.md diff --git a/README.ad b/README.ad new file mode 100644 index 0000000..a1788bd --- /dev/null +++ b/README.ad @@ -0,0 +1,26 @@ +# Draft FreeBSD Git docs + +This repo contains the draft docs for the FreeBSD migration to Git. + +## How to read the docs + +Please check out the link:SUMMARY.md[SUMMARY] page for a clean and complete list of the documentation. + +The best place to start is with the link:mini-primer.md[FreeBSD mini-git Primer] if +you are coming to this fresh. Otherwise the best place to start is the +link:src-cvt.md[FreeBSD Src Committer Transition Guide] if you have an existing source tree +from the project you'd like to convert over to the new git repo. Once you've +read that, you can read the link:mini-primer.md[FreeBSD mini-git Primer] for a +general primer. + +More advanced topics can be found in the link:faq.md[Git FAQ]. + +Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. + +## Contributing + +Please feel free to submit pull requests for typos, clearer language, additional +topics, etc. + +Please open issues for something that you're just noting the problem, but +don't have a solution for. diff --git a/README.md b/README.md deleted file mode 100644 index ece9865..0000000 --- a/README.md +++ /dev/null @@ -1,21 +0,0 @@ -# Draft FreeBSD Git docs - -This repo contains the draft docs for the FreeBSD migration to Git. - -## How to read the docs - -Please check out the [Summary](SUMMARY.md) page for a clean and complete list of the documentation. - -The best place to start is with the [FreeBSD mini-git Primer](mini-primer.md) if you are coming to this fresh. Otherwise the best place to start is the [FreeBSD Src Committer Transition Guide](src-cvt.md) if you have an existing source tree from the project you'd like to convert over to the new git repo. Once you've read that, you can read the [FreeBSD mini-git Primer](mini-primer.md) for a general primer. - -More advanced topics can be found in the [Git FAQ](faq.md). - -Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. - -## Contributing - -Please feel free to submit pull requests for typos, clearer language, additional -topics, etc. - -Please open issues for something that you're just noting the problem, but -don't have a solution for. From 02ebf7295fc19a8a865ea91c804995f5610e3e7b Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Tue, 2 Feb 2021 11:51:18 -0700 Subject: [PATCH 25/44] Better? --- README.ad => README.adoc | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename README.ad => README.adoc (100%) diff --git a/README.ad b/README.adoc similarity index 100% rename from README.ad rename to README.adoc From f67331a9e1a82a4acea0eab4937e3910c5246cbc Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Tue, 2 Feb 2021 11:54:40 -0700 Subject: [PATCH 26/44] Better? --- README.adoc | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/README.adoc b/README.adoc index a1788bd..df2789c 100644 --- a/README.adoc +++ b/README.adoc @@ -1,8 +1,18 @@ -# Draft FreeBSD Git docs += Draft FreeBSD Git docs +:doctype: article +:description: A first draft of FreeBSD Git docs +:author: Warner Losh +:email: imp@FreeBSD.org +:source-highlighter: rouge +:rouge-style: github +:!showtitle: +:icons: font +:toc: preamble + This repo contains the draft docs for the FreeBSD migration to Git. -## How to read the docs +== How to read the docs Please check out the link:SUMMARY.md[SUMMARY] page for a clean and complete list of the documentation. @@ -17,7 +27,7 @@ More advanced topics can be found in the link:faq.md[Git FAQ]. Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. -## Contributing +== Contributing Please feel free to submit pull requests for typos, clearer language, additional topics, etc. From e64cdec0ad90cdb4fe7f430a2586cb2ab3b1d435 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Tue, 2 Feb 2021 11:55:39 -0700 Subject: [PATCH 27/44] show title --- README.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.adoc b/README.adoc index df2789c..439f8e2 100644 --- a/README.adoc +++ b/README.adoc @@ -1,11 +1,11 @@ = Draft FreeBSD Git docs + :doctype: article :description: A first draft of FreeBSD Git docs :author: Warner Losh :email: imp@FreeBSD.org :source-highlighter: rouge :rouge-style: github -:!showtitle: :icons: font :toc: preamble From 59971bf702f7692cea36a4c70aab575c7fe04cc0 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Thu, 4 Feb 2021 10:29:16 -0700 Subject: [PATCH 28/44] Move summary to asciidoc --- README.adoc | 2 +- SUMMARY.adoc | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++ SUMMARY.md | 44 --------------------------------------- 3 files changed, 59 insertions(+), 45 deletions(-) create mode 100644 SUMMARY.adoc delete mode 100644 SUMMARY.md diff --git a/README.adoc b/README.adoc index 439f8e2..d2c9dff 100644 --- a/README.adoc +++ b/README.adoc @@ -14,7 +14,7 @@ This repo contains the draft docs for the FreeBSD migration to Git. == How to read the docs -Please check out the link:SUMMARY.md[SUMMARY] page for a clean and complete list of the documentation. +Please check out the link:SUMMARY.adoc[SUMMARY] page for a clean and complete list of the documentation. The best place to start is with the link:mini-primer.md[FreeBSD mini-git Primer] if you are coming to this fresh. Otherwise the best place to start is the diff --git a/SUMMARY.adoc b/SUMMARY.adoc new file mode 100644 index 0000000..d5ad47b --- /dev/null +++ b/SUMMARY.adoc @@ -0,0 +1,58 @@ +# Summary + +:doctype: article +:description: A first draft of FreeBSD Git docs +:author: Warner Losh +:email: imp@FreeBSD.org +:source-highlighter: rouge +:rouge-style: github +:icons: font +:toc: preamble + +== Preface + +* link:git-why.md[FreeBSD moving to Git: Why] + +== Getting Started + +* link:git-how.md[How FreeBSD implements Git] +* link:mini-primer.md[FreeBSD mini-git Primer] +* link:URLs.md[Important URLs of the FreeBSD Git Infrastructure] + +== `doc` Specific + +* link:doc-cvt.md[FreeBSD Doc Committer Transition Guide] + +== `src` Specific + +* link:src-cvt.md[FreeBSD Src Committer Transition Guide] +* link:vendor.md[(WIP) How to do vendor imports] +* link:MFC.md[(WIP) How to MFC] +* link:projects-user.md[A brief note about Subversion Projects and User branches] + +== Git References + +* link:faq.md[Git FAQ] + +== Committer References + +* link:commit.md[Writing Good FreeBSD Commit Messages] +* link:meta.md[Include appropriate metadata in a footer] +* link:pull-request.md[How to deal with Pull Requests] + +== Design & Implementation + +* link:big-picture.md[Big Picture] +* link:design-notes.md[Design Notes] + +== Next Steps + +* link:github-mirroring.md[Github mirroring plan] + +== Contributing + +Please feel free to submit pull requests for typos, clearer language, additional +topics, etc. + +Please open issues for something that you're just noting the problem, but +don't have a solution for. diff --git a/SUMMARY.md b/SUMMARY.md deleted file mode 100644 index 9d89ef5..0000000 --- a/SUMMARY.md +++ /dev/null @@ -1,44 +0,0 @@ -# Summary - -## Preface - -* [README](README.md) -* [FreeBSD moving to Git: Why](git-why.md) - -## Getting Started - -* [How FreeBSD implements Git](git-how.md) -* [FreeBSD mini-git Primer](mini-primer.md) -* [Important URLs of the FreeBSD Git Infrastructure](URLs.md) - -## `doc` Specific - -* [FreeBSD Doc Committer Transition Guide](doc-cvt.md) -* [~~FreeBSD Doc tree migration schedule~~](doc-cutover-schedule.md) (Migration Completed as of Dec 9, 2020) -* [Todo List for -doc](doc-todo.md) - -## `src` Specific - -* [FreeBSD Src Committer Transition Guide](src-cvt.md) -* [(WIP) How to do vendor imports](vendor.md) -* [(WIP) How to MFC](MFC.md) -* [A brief note about Subversion Projects and User branches](projects-user.md) - -## Git References - -* [Git FAQ](faq.md) - -## Committer References - -* [Writing Good FreeBSD Commit Messages](commit.md) -* [Include appropriate metadata in a footer](meta.md) -* [How to deal with Pull Requests](pull-request.md) - -## Design & Implementation - -* [Big Picture](big-picture.md) -* [Design Notes](design-notes.md) - -## Next Steps - -* [Github mirroring plan](github-mirroring.md) From a5ac8b101d28ef420f637dfda499e76ac6c71c75 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Thu, 4 Feb 2021 10:31:45 -0700 Subject: [PATCH 29/44] Fold in readme info --- SUMMARY.adoc | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/SUMMARY.adoc b/SUMMARY.adoc index d5ad47b..01dfe50 100644 --- a/SUMMARY.adoc +++ b/SUMMARY.adoc @@ -49,6 +49,19 @@ * link:github-mirroring.md[Github mirroring plan] +== How to read the docs + +The best place to start is with the link:mini-primer.md[FreeBSD mini-git Primer] if +you are coming to this fresh. Otherwise the best place to start is the +link:src-cvt.md[FreeBSD Src Committer Transition Guide] if you have an existing source tree +from the project you'd like to convert over to the new git repo. Once you've +read that, you can read the link:mini-primer.md[FreeBSD mini-git Primer] for a +general primer. + +More advanced topics can be found in the link:faq.md[Git FAQ]. + +Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. + == Contributing Please feel free to submit pull requests for typos, clearer language, additional From 096afef4a8b9d8964f9edaf3d15e484b1ead83d3 Mon Sep 17 00:00:00 2001 From: Li-Wen Hsu Date: Fri, 5 Feb 2021 03:28:24 +0800 Subject: [PATCH 30/44] Mention using worktree for the internal/admin branch --- URLs.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/URLs.md b/URLs.md index 24daffc..3169bb1 100644 --- a/URLs.md +++ b/URLs.md @@ -124,6 +124,10 @@ git config --add remote.freebsd.fetch '+refs/internal/*:refs/internal/*' git fetch git checkout -b admin internal/admin ``` +Alternatively, you can add a worktree for the `admin` branch: +``` +git worktree add -b admin ../${repo}-admin internal/admin +``` For browsing `internal/admin` branch on web: https://cgit.freebsd.org/${repo}/log/?h=internal/admin From 41c566e90206ae1bf9951ba326ae680d4f9d5a6c Mon Sep 17 00:00:00 2001 From: Li-Wen Hsu Date: Mon, 8 Feb 2021 15:36:24 +0800 Subject: [PATCH 31/44] Add a note that committers don't need to change the git user in ssh URL --- URLs.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/URLs.md b/URLs.md index 3169bb1..100138d 100644 --- a/URLs.md +++ b/URLs.md @@ -13,13 +13,14 @@ Following is the quick map for the essential URLs from Subversion to Git: | ---------------------------------------- | ------------------------------- | ----------------------------------- | | Web-based repository browser | `https://svnweb.freebsd.org/${repo}` | `https://cgit.freebsd.org/${repo}` | | Distributed mirrors for anonymous read-only checkout/clone | `https://svn.freebsd.org/${repo}` `svn://svn.freebsd.org/${repo}` | `https://git.freebsd.org/${repo}.git` `ssh://anongit@git.freebsd.org/${repo}.git` | -| Read/write Repository for committers (**) | `svn+ssh://(svn)repo.freebsd.org/${repo}` | `ssh://git@(git)repo.freebsd.org/${repo}.git` | +| Read/write Repository for committers (**) | `svn+ssh://(svn)repo.freebsd.org/${repo}` | `ssh://git@(git)repo.freebsd.org/${repo}.git` (***) | - (*) In subversion, `base` is used for FreeBSD src repository (`src`). - (**) Before all repositories in SVN have been migrated, the URL of the read-write repository, repo.freebsd.org, will be pointing to one of: - svnrepo.freebsd.org - gitrepo.freebsd.org + - (***) `git` is a special user on the repository server which will map your registered ssh key in FreeBSD.org to your identity, no need to change it. Please use the hostname that explicitly includes the VCS name to access the right repositories during the migration. `repo.freebsd.org` From 2f6c1f6132af2a60587a55a9ec6f4e25d2f33689 Mon Sep 17 00:00:00 2001 From: Li-Wen Hsu Date: Mon, 8 Feb 2021 17:27:08 +0800 Subject: [PATCH 32/44] Clarify the hostname more --- URLs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/URLs.md b/URLs.md index 100138d..fad2086 100644 --- a/URLs.md +++ b/URLs.md @@ -22,7 +22,7 @@ Following is the quick map for the essential URLs from Subversion to Git: - gitrepo.freebsd.org - (***) `git` is a special user on the repository server which will map your registered ssh key in FreeBSD.org to your identity, no need to change it. - Please use the hostname that explicitly includes the VCS name to + Please use the hostname that explicitly includes the VCS name (ex: `gitrepo` or `svnrepo`) to access the right repositories during the migration. `repo.freebsd.org` will be the canonical FreeBSD Git repository for the committers after all the repositories migrated to Git. From 235285e96853a76d845a1190fe481deed8f502aa Mon Sep 17 00:00:00 2001 From: Ed Maste Date: Sat, 20 Feb 2021 14:05:48 -0500 Subject: [PATCH 33/44] Replace WIP commit message doc with link to committed version --- commit.md | 177 +----------------------------------------------------- 1 file changed, 1 insertion(+), 176 deletions(-) diff --git a/commit.md b/commit.md index dfb4220..d0e119f 100644 --- a/commit.md +++ b/commit.md @@ -1,178 +1,3 @@ -[[ From an article Ed Maste wrote ]] - # Writing Good FreeBSD Commit Messages -## Why are commit messages important? - -When you commit a change in Git, Subversion, or another version -control system (VCS), you're prompted to write some text describing -the commit -- a commit message. How important is this commit message? -Should you spend some significant effort writing it? Does it really -matter if you write simply `fixed a bug`? - -Most projects have more than one developer and last for some length of -time. Commit messages are a very important method of communicating -with other developers, in the present and for the future. - -FreeBSD has hundreds of active developers and hundreds of thousands of -commits spanning decades of history. Over that time the developer -community has learned how valuable good commit messages are; sometimes -these are hard-learned lessons. - -Commit messages serve at least three purposes: - -### Communicating with other developers - -FreeBSD commits generate email to various mailing lists. These include -the commit message along with a copy of the patch itself. Commit -messages are also viewed through commands like `git log`. These serve -to make other developers aware of changes that are ongoing; that other -developer may want to test the change, may have an interest in the -topic and will want to review in more detail, or may have their own -projects underway that would benefit from interaction. - -### Making Changes Discoverable - -In a large project with a long history it may be difficult to find -changes of interest when investigating an issue or change in -behaviour. Verbose, detailed commit messages allow searches for -changes that might be relevant. For example, `git log --since 1year ---grep 'USB timeout'`. - -### Providing historical documentation - -Commit messages serve to document changes for future developers, -perhaps years or decades later. This future developer may even be you, -the original author. A change that seems obvious today may be -decidedly not so much later on. - -The `git blame` command annotates each line of a source file with the -change (hash and subject line) that brought it in. - -Having established the importance, here are elements of a good FreeBSD -commit message: - -1. Start with a subject line - -Commit messages should start with a single-line subject that briefly -summarizes the change. The subject should, by itself, allow the reader -to quickly determine if the change is of interest or not. - -2. Keep subject lines short - -The subject line should be as short as possible while still retaining -the required information. This is to make browsing `git log` more -efficient, and so that `git log --oneline` can display the short hash -and subject on a single 80-column line. A good rule of thumb is to -stay below 63 characters, and aim for about 50 or fewer if possible. - -3. Prefix the subject line with a component, if applicable - -If the change relates to a specific component the subject line may be -prefixed with that component name and a colon (:). - -✓ `foo: Add -k option to keep temporary data` - -Include the prefix in the 63-character limit suggested above, so that -`git log --oneline` avoids wrapping. - -4. Capitalize the first letter of the subject - -Capitalize the first letter of the subject itself. The prefix, if any, -is not capitalized unless necessary (e.g., `USB:`). - -5. Do not end the subject line with punctuation - -Do not end with a period or other punctuation. In this regard the -subject line is like a newspaper headline. - -6. Separate the subject and body with a blank line - -Separate the body from the subject with a blank line. - -Some trivial commits do not require a body, and will have only a subject. - -✓ `ls: fix typo in usage text` - -7. Limit messages to 72 columns - -`git log` and `git format-patch` indent the commit message by four -spaces. Wrapping at 72 columns provides a matching margin on the right -edge. Limiting messages to 72 characters also keeps the commit message -in formatted patches below RFC 2822's suggested email line length -limit of 78 characters. This limit works well with a variety of tools -that may render commit messages; line wrapping might be inconsistent -with longer line length. - -8. Use the present tense, imperative mood - -This facilitates short subject lines and provides consistency, -including with automatically generated commit messages (e.g., as -generated by `git revert`). This is important when reading a list of -commit subjects. Think of the subject as finishing the sentence "when -applied, this change will ...". - -✓ `foo: Implement the -k (keep) option` -✗ `foo: Implemented the -k option` -✗ `This change implements the -k option in foo` -✗ `-k option added` - -9. Focus on what and why, not how - -Explain what the change accomplishes and why it is being done, rather -than how. - -Do not assume that the reader is familiar with the issue. Explain the -background and motivation for the change. Include benchmark data if -you have it. - -If there are limitations or incomplete aspects of the change, describe them in the commit message. - -10. Consider whether parts of the commit message could be code comments instead - -Sometimes while writing a commit message you may find yourself writing -a sentence or two explaining some tricky or confusing aspect of the -change. When this happens consider whether it would be valuable to -have that explanation as a comment in the code itself. - -11. Write commit messages for your future self - -While writing the commit message for a change you have all of the -context in mind - what prompted the change, alternate approaches that -were considered and rejected, limitations of the change, and so -on. Imagine yourself revisiting the change a year or two in the -future, and write the commit message in a way that would provide that -necessary context. - -12. Commit messages should stand alone - -You may include references to mailing list postings, benchmark result web sites, or code review links. However, the commit message should contain all of the relevant information in case these references are no longer available in the future. - -Similarly, a commit may refer to a previous commit, for example in the case of a bug fix or revert. In addition to the commit identifier (revision or hash), include the subject line from the referenced commit (or another suitable brief reference). With each VCS migration (from CVS to Subversion to Git) revision identifiers from previous systems may become difficult to follow. - -13. Include appropriate metadata in a footer - -As described in the metadata standard. - -14. Take the opportunity to reconsider multiple commits - -# Other notes, not for commit messages: - -References: -- https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/commit-log-message.html -- https://www.conventionalcommits.org/en/v1.0.0/ -- https://pyinstaller.readthedocs.io/en/v3.3.1/development/commit-messages.html -- https://wpvip.com/documentation/commit-messages/ -- https://chris.beams.io/posts/git-commit/ -- https://stackoverflow.com/questions/2290016/git-commit-messages-50-72-formatting -https://gist.github.com/robertpainsi/b632364184e70900af4ab688decf6f53 -https://wiki.openstack.org/wiki/GitCommitMessages#Information_in_commit_messages -https://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html -https://medium.com/swlh/why-should-i-write-good-commit-messages-e15d37bf45cb -https://www.freecodecamp.org/news/writing-good-commit-messages-a-practical-guide/ -https://dev.to/cvortmann/what-makes-a-good-commit-message-181i - -## Avoid combining logically distinct changes into one commit - -## Avoid combining whitespace or style changes with functional changes - +Now available in the [committer's guide](https://docs.freebsd.org/en/articles/committers-guide/#commit-log-message). From 09644252cc3305a0a0b1d5ceadf32b9886d48cc7 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Sat, 27 Feb 2021 00:23:52 -0700 Subject: [PATCH 34/44] Update mini-primer to note restrictions --- mini-primer.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/mini-primer.md b/mini-primer.md index 2ad20da..a6da132 100644 --- a/mini-primer.md +++ b/mini-primer.md @@ -77,20 +77,21 @@ versions of history). ### Shallow Clone -A shallow clone copies just the most current code, but none or little -of the history. This can be useful when you need to build a specific -revision of FreeBSD, or when you are just starting out and plan to -track the tree more fully. You can also use it to limit history to -only so many revisions. +A shallow clone copies just the most current code, but none or little of the history. +This can be useful when you need to build a specific revision of FreeBSD, or when you are just starting out and plan to track the tree more fully. +You can also use it to limit history to only so many revisions. +However, see below for a significant limitation of this approach. ``` % git clone -o freebsd -b branch --depth 1 $URL [dir] ``` -This clones the repository, but only has the most recent version in -the repository. The rest of the history is not downloaded. Should you -change your mind later, you can do 'git fetch --unshallow' to get the -old history. +This clones the repository, but only has the most recent version in the repository. +The rest of the history is not downloaded. +Should you change your mind later, you can do 'git fetch --unshallow' to get the old history. + +WARNING: When you make a shallow clone, you will lose the commit count in your uname output. +This can make it more difficult to determine if your system needs to be updated when a security advisory is issued. ### Building From fccae31b99d15a33c14579943d66cb8da9e34329 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Sat, 27 Feb 2021 00:25:10 -0700 Subject: [PATCH 35/44] Doc cutover is obsolete -- remove it --- doc-cutover-schedule.md | 57 ----------------------------------------- 1 file changed, 57 deletions(-) delete mode 100644 doc-cutover-schedule.md diff --git a/doc-cutover-schedule.md b/doc-cutover-schedule.md deleted file mode 100644 index d905e95..0000000 --- a/doc-cutover-schedule.md +++ /dev/null @@ -1,57 +0,0 @@ -# FreeBSD Doc tree conversion - -The FreeBSD project will be converting from Subversion to Git in the -coming weeks and months. The doc repo will be converted starting -December 5th. This document provides a detailed schedule. - -## Timetable - -Note: specific times are subject to change as the detailed logistics are worked out. - -| time (UTC) | who | what | -| ------------- | ------- | ---------------------------------------------------------- | -| Dec 1st 23:00 | imp | ~Send mail to community about cutover~ | -| Dec 2nd 23:59 | so@ | ~Last Advisory before cut over~ | -| Dec 4th 0:00 | re@ | ~Last snapshot before cut over starts~ | -| Dec 4th 16:00 | uqs | ~Freeze hashes for freebsd-doc~ | -| Dec 4th 16:01 | lwhsu | ~Finalize Git repo at freebsd.org repo~ | -| Dec 4th 23:30 | gjb | ~Commit updates to webupdate/webupdate.wrapper to SVN~ | -| Dec 5th 0:00 | gjb | ~Start switch website / handbook building from SVN to Git~ | -| Dec 5th 12:00 | gjb | ~GO/NOGO on switch finalization~ | -| Dec 7th 22:00 | uqs | ~Turn off scheduled SVN -> Git converter~ | -| Dec 8th 2:59 | lwhsu | ~Make a final commit to Subversion~ | -| Dec 8th 3:00 | lwhsu | ~Turn off write access to Subversion~ | -| Dec 8th 3:01 | lwhsu | ~Snapshot the Subversion repository repo filesystem~ | -| Dec 9th 9:00 | uqs | ~Start final run of SVN to Git converter~ | -| Dec 9th 10:00 | uqs | Push converted tree to GitHub/GitLab | -| Dec 9th 11:00 | lwhsu | ~Turn on push to Git~ | -| Dec 9th 11:01 | lwhsu | ~Push 'Welcome to Git' commit~ | -| Dec 9th 12:00 | so@ | ~Next advisory window opens~ | -| Dec 11th 0:00 | re@ | ~Next snapshot starts~ | - -## Open Issues - -1. What about GitLab? (bapt?) -2. Final plan for GitHub conversion (imp) -3. Finish documentation on new doc workflow -- in progress (imp) - -Announcement draft: https://hackmd.io/0r0OnsTjRTyojFeDpJZBng - -## Old vs New URL translation table - -SVN infra -> Git infra map - -| Item | SVN | GIT | -| ---------------------------------------- | ------------------------------- | ----------------------------------- | -| Web-based repository browser | https://svnweb.freebsd.org | https://cgit.freebsd.org | -| Distributed mirrors for anonymous readonly checkout/clone | https://svn.freebsd.org svn://svn.freebsd.org | https://git.freebsd.org ssh://anongit@git.freebsd.org | -| Read/write Repository for committers (*) | svn+ssh://(svn)repo.freebsd.org | ssh://git@(git)repo.freebsd.org | - -(*) Before all repositories in SVN have been migrated, the repo.freebsd.org will be pointing to one of: - - svnrepo.freebsd.org - - gitrepo.freebsd.org - -please use the hostname that explicitly includes the VCS name to -access the right repositories during the migration. `repo.freebsd.org` -will be the canonical FreeBSD Git repository for the committers after -all the repositories migrated to Git. From 71acc146ceeede3dc383c4f4c0634032bdc97b9f Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Sat, 27 Feb 2021 00:27:40 -0700 Subject: [PATCH 36/44] Update for a post-cutover world --- mini-primer.md | 25 +++++-------------------- 1 file changed, 5 insertions(+), 20 deletions(-) diff --git a/mini-primer.md b/mini-primer.md index a6da132..015db67 100644 --- a/mini-primer.md +++ b/mini-primer.md @@ -32,26 +32,11 @@ is the default branch if you omit the '-b branch' or '--branch branch' options below. ### Repositories -At the moment, there are two repositories: - -- The old GitHub repository (https://github.com/freebsd/freebsd.git) - was exported by the project for several years to support downstream - users using git. There were a large number of mistakes in the repo - that have led us to generate a new export to be the source of truth, - which will as a result have new hashes. - -- The new git repo (https://cgit.freebsd.org/src.git) is - the authoritative repository (and history) - for the project. Its cloning URLs are https://git.freebsd.org/src.git, - ssh://anongit@git.FreeBSD.org/src.git or - ssh://git@gitrepo.FreeBSD.org/src.git (developers with SSH key only). - -The new repository should be used as $URL in the commands below. - -Note: The project doesn't use submodules as they are a poor fit for -our workflows and development model. How we track changes in -third-party applications is discussed elsewhere and generally of -little concern to the casual user. +Please see [URLs.md] for the latest information on where to get FreeBSD soruces. +$URL below can be obstained from that page. + +Note: The project doesn't use submodules as they are a poor fit for our workflows and development model. +How we track changes in third-party applications is discussed elsewhere and generally of little concern to the casual user. ### Deep Clone A deep clone pulls in the entire tree, as well as all the history and From c3f403acaa1003a47ad597b60d25be280a6e9607 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Fri, 12 Mar 2021 14:04:51 -0700 Subject: [PATCH 37/44] reformat to one sentence one line --- design-notes.md | 55 ++++++++++++++++++------------------------------- 1 file changed, 20 insertions(+), 35 deletions(-) diff --git a/design-notes.md b/design-notes.md index 59509c4..5365815 100644 --- a/design-notes.md +++ b/design-notes.md @@ -1,9 +1,9 @@ # Design notes -I've abstracted the README from git_conv here. It was written by -uqs@. This is still quite rough, but contains useful information, even -in its currently lightly-edited form from there. More revision will be -forthcoming. +I've abstracted the README from git_conv here. +It was written by uqs@. +This is still quite rough, but contains useful information, even in its currently lightly-edited form from there. +More revision will be forthcoming. ## Gimme the repo! @@ -12,16 +12,12 @@ git clone https://cgit-beta.freebsd.org/src.git && cd src git config --add remote.freebsd.fetch '+refs/notes/*:refs/notes/*' && git fetch ``` -Same for the `doc` and `ports` repos. You can expect some things to be -different. Most importantly you should check whether branch and mergepoints -(especially for vendor branches, but also project branches) are there and make -sense. +Same for the `doc` and `ports` repos. You can expect some things to be different. +Most importantly you should check whether branch and mergepoints (project branches) are there and make sense. Neither `vendor`, `user` or `projects` branches are "visible" by default. -`backups` refs are deleted branches and `cvs2svn` contains some of the detritus -left over from the CVS days and the cvs2svn conversion. The `internal` -namespace for now only has the `access` and `mentors` file, detailing when -people got their various commit bits. +`backups` refs are deleted branches and `cvs2svn` contains some of the detritus left over from the CVS days and the cvs2svn conversion. +The `internal` namespace for now only has the `access` and `mentors` file, detailing when people got their various commit bits. ``` git config --add remote.freebsd.fetch '+refs/vendor/*:refs/vendor/*' @@ -33,32 +29,22 @@ git config --add remote.freebsd.fetch '+refs/internal/*:refs/internal/*' git fetch ``` -Note that `internal`, `projects` and `user` branches also exist for the `doc` -repo and `ports` has `projects` and `releng` as well. vendor, cvs2svn and -backups are exclusive to the `src` repo though. +Note that `internal`, `projects` and `user` branches also exist for the `doc` repo and `ports` has `projects` and `releng` as well. +cvs2svn and backups are exclusive to the `src` repo though. - `user/` branches are never merged back into `master` -- MFHs into `user/` or `projects/` branches are just cherry-picks to keep `git - log --graph` somewhat readable and as merges wouldn't convey any useful - information, really. -- `vendor` **tags** were never flattened post-creation, as that would advance - them off of the mainline branch and make them invisible to a simple `git log` -- `release/1.0_cvs` et al. are snapshots of the checked out CVS source - code, including expanded $Id$ tags. -- no other keywords are expanded -- various vendor-foo suffixes have been collapsed into 1 vendor namespace, - except for a few vendors where merging the userland and kernel bits is not - straightforward due to how they interleave with the merge and branch history. -- some branches have their history "extended", that is, commits under the - `cvs2svn` area were properly attached. -- ... and most of these commits have actually been inlined directly into the - mainline tree to keep the history more "linear" and associate the commit with - the original author and commit message. +- MFHs into `user/` or `projects/` branches are just cherry-picks to keep `git log --graph` somewhat readable and as merges wouldn't convey any useful information, really. +- `vendor` **tags** were never flattened post-creation, as that would advance them off of the mainline branch and make them invisible to a simple `git log` +- `release/1.0_cvs` et al. are snapshots of the checked out CVS source code, including expanded $Id$ tags. +- no other keywords are expanded. +- various vendor-foo suffixes have been collapsed into 1 vendor namespace, except for a few vendors where merging the userland and kernel bits is not straightforward due to how they interleave with the merge and branch history. +- some branches have their history "extended", that is, commits under the `cvs2svn` area were properly attached. +- ... and most of these commits have actually been inlined directly into the mainline tree to keep the history more "linear" and associate the commit with the original author and commit message. ## How to analyze the results -Here are some tips that were deemed useful in making sense of resulting -history. Please add a "graph notes log" alias to your `.gitconfig`: +Here are some tips that were deemed useful in making sense of resulting history. +Please add a "graph notes log" alias to your `.gitconfig`: ``` [alias] gnlog = log --graph --pretty=format:'%Cred%h %C(green)%t %Creset %C(red)%ad %Creset-%C(yellow)%d%Creset %s %n %N %-GG' --date=short @@ -81,8 +67,7 @@ git show -p `git log --format=%h --notes --grep=revision=294706$` ``` git gnlog vendor/zstd/dist master ``` -(but you'll need to search in the massive output for where the vendor branch is -being merged, if you know a better way to represent this, please let us know!) +(but you'll need to search in the massive output for where the vendor branch is being merged, if you know a better way to represent this, please let us know!) ### Look for commits with more than 5 parents and log them From 530dcda3cd89c3334d128def145d3e4c02e6ff0b Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Fri, 12 Mar 2021 14:06:43 -0700 Subject: [PATCH 38/44] doc todo is done --- doc-todo.md | 3 --- 1 file changed, 3 deletions(-) delete mode 100644 doc-todo.md diff --git a/doc-todo.md b/doc-todo.md deleted file mode 100644 index 52d16dc..0000000 --- a/doc-todo.md +++ /dev/null @@ -1,3 +0,0 @@ -# Simple todo list for docs - -[ ] Need to document how to document freebsd rev bumps in handbook From 32614295e038243cf8b41135f22b4331dbe0770f Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Sun, 14 Mar 2021 17:55:27 -0600 Subject: [PATCH 39/44] Update for stuff moved into the handbook --- SUMMARY.adoc | 6 +- URLs.md | 175 -------------------------------------------- github-mirroring.md | 130 -------------------------------- 3 files changed, 1 insertion(+), 310 deletions(-) delete mode 100644 URLs.md delete mode 100644 github-mirroring.md diff --git a/SUMMARY.adoc b/SUMMARY.adoc index 01dfe50..4a9bea5 100644 --- a/SUMMARY.adoc +++ b/SUMMARY.adoc @@ -17,7 +17,7 @@ * link:git-how.md[How FreeBSD implements Git] * link:mini-primer.md[FreeBSD mini-git Primer] -* link:URLs.md[Important URLs of the FreeBSD Git Infrastructure] +* https://docs.freebsd.org/en/books/handbook/mirrors/#git[Important URLs of the FreeBSD Git Infrastructure] == `doc` Specific @@ -45,10 +45,6 @@ * link:big-picture.md[Big Picture] * link:design-notes.md[Design Notes] -== Next Steps - -* link:github-mirroring.md[Github mirroring plan] - == How to read the docs The best place to start is with the link:mini-primer.md[FreeBSD mini-git Primer] if diff --git a/URLs.md b/URLs.md deleted file mode 100644 index fad2086..0000000 --- a/URLs.md +++ /dev/null @@ -1,175 +0,0 @@ -# Important URLs of the FreeBSD Git Infrastructure - -This document contains the important URLs of the Git infrastructure of the -FreeBSD project, and the mapping for the Subversion infrastructure. - -## Old vs New URL translation table - -Following is the quick map for the essential URLs from Subversion to Git: - -`${repo}` can be replaced by `doc`, `ports` and `src`. (*) - -| Item | Subversion | Git | -| ---------------------------------------- | ------------------------------- | ----------------------------------- | -| Web-based repository browser | `https://svnweb.freebsd.org/${repo}` | `https://cgit.freebsd.org/${repo}` | -| Distributed mirrors for anonymous read-only checkout/clone | `https://svn.freebsd.org/${repo}` `svn://svn.freebsd.org/${repo}` | `https://git.freebsd.org/${repo}.git` `ssh://anongit@git.freebsd.org/${repo}.git` | -| Read/write Repository for committers (**) | `svn+ssh://(svn)repo.freebsd.org/${repo}` | `ssh://git@(git)repo.freebsd.org/${repo}.git` (***) | - - - (*) In subversion, `base` is used for FreeBSD src repository (`src`). - - (**) Before all repositories in SVN have been migrated, the URL of the - read-write repository, repo.freebsd.org, will be pointing to one of: - - svnrepo.freebsd.org - - gitrepo.freebsd.org - - (***) `git` is a special user on the repository server which will map your registered ssh key in FreeBSD.org to your identity, no need to change it. - - Please use the hostname that explicitly includes the VCS name (ex: `gitrepo` or `svnrepo`) to - access the right repositories during the migration. `repo.freebsd.org` - will be the canonical FreeBSD Git repository for the committers after - all the repositories migrated to Git. - -There are also external mirrors maintained by project members available, please refer to "External mirrors" section. - -### SSH related information - - - `ssh://${user}@${url}/${repo}.git` can be written as `${user}@${url}:${repo}.git`, i.e., following two URLs are both valid for passing to git: - - `ssh://anongit@git.freebsd.org/${repo}.git` - - `anongit@git.freebsd.org:${repo}.git` - - As well as the read-write repo: - - `ssh://git@(git)repo.freebsd.org/${repo}.git` - - `git@(git)repo.freebsd.org:${repo}.git` - -- gitrepo.FreeBSD.org host key fingerprints: - - - ECDSA key fingerprint is `SHA256:seWO5D27ySURcx4bknTNKlC1mgai0whP443PAKEvvZA` - - ED25519 key fingerprint is `SHA256:lNR6i4BEOaaUhmDHBA1WJsO7H3KtvjE2r5q4sOxtIWo` - - RSA key fingerprint is `SHA256:f453CUEFXEJAXlKeEHV+ajJfeEfx9MdKQUD7lIscnQI` - -- git.FreeBSD.org host key fingerprints: - - - ECDSA key fingerprint is `SHA256:/UlirUAsGiitupxmtsn7f9b7zCWd0vCs4Yo/tpVWP9w` - - ED25519 key fingerprint is `SHA256:y1ljKrKMD3lDObRUG3xJ9gXwEIuqnh306tSyFd1tuZE` - - RSA key fingerprint is `SHA256:jBe6FQGoH4HjvrIVM23dcnLZk9kmpdezR/CvQzm7rJM` - -These are also published as SSHFP records in DNS. - - -## Web-based repository browser - -The FreeBSD project currently uses cgit as the web-based repository browser: https://cgit.freebsd.org/ -The URL of the indivirual repository is at: https://cgit.freebsd.org/${repo}/ - -## For Users - -Using `git clone` and `git pull` from the official distributed mirros is recommended. The GeoDNS should direct you to the nearest mirror to you. - -## For Developers - -This section describes the read-write access for committers to push the commits from develoeprs or contributors. For read-only access, please refer to the users section above. - -We only document the important URLs here, the full information of retrieving all data in the repository is available at: -https://github.com/freebsd/git_conv#gimme-the-repo - -### Daily use - -* Clone the repository: - ``` - git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/${repo}.git - ``` - Then you should have the official mirrors as your remote: - ``` - $ git remote -v - freebsd https://git.freebsd.org/${repo}.git (fetch) - freebsd https://git.freebsd.org/${repo}.git (push) - ``` - -* Config the FreeBSD committer data: - - The commit hook in repo.freebsd.org checks the "Commit" field matches the - committer's information in FreeBSD.org. The easiest way to get the suggested - config is by executing `/usr/local/bin/gen-gitconfig.sh` script on freefall: - - ``` - $ gen-gitconfig.sh - [...] - git config user.name (your name in gecos) - git config user.email (your login)FreeBSD.org - ```` - -* Set the push URL: - ``` - $ git remote set-url --push freebsd git@gitrepo.freebsd.org:${repo}.git - ``` - Then you should have separated fetch and push URLs as the most efficient setup: - ``` - $ git remote -v - freebsd https://git.freebsd.org/${repo}.git (fetch) - freebsd git@gitrepo.freebsd.org:${repo}.git (push) - ``` - Again, note that `gitrepo.freebsd.org` will be canonicalized to `repo.freebsd.org` in the future. - -* Install commit message template hook: - ``` - fetch https://cgit.freebsd.org/src/plain/tools/tools/git/hooks/prepare-commit-msg -o .git/hooks - chmod 755 .git/hooks/prepare-commit-msg - ``` - -### "admin" branch - -The `access` and `mentors` files are stored in an orphan branch, `internal/admin`, in each repository. - -Following example is how to check out the `internal/admin` branch to a local branch named `admin`: - -``` -git config --add remote.freebsd.fetch '+refs/internal/*:refs/internal/*' -git fetch -git checkout -b admin internal/admin -``` -Alternatively, you can add a worktree for the `admin` branch: -``` -git worktree add -b admin ../${repo}-admin internal/admin -``` - -For browsing `internal/admin` branch on web: -https://cgit.freebsd.org/${repo}/log/?h=internal/admin - -For pushing, either specify the full refspec: -``` -git push freebsd HEAD:refs/internal/admin -``` - -Or setting `push.default` to `upstream` which will make `git push` to push the current branch back to its upstream by default, which is more suitable for our workflow: -``` -git config push.default upstream -``` - -## External mirrors - -Those mirrors are not hosted in FreeBSD.org but still maintained by the project members. -Users and developers are welcomed to pull or browse repositories on those mirrors. -The project workflow with those mirrors are still under discussion. - -### Codeberg - - doc: https://codeberg.org/FreeBSD/freebsd-doc - -### GitHub - - doc: https://github.com/freebsd/freebsd-doc - -### GitLab - - doc: https://gitlab.com/FreeBSD/freebsd-doc - -## Mailing lists - -General usage and questions about git in the FreeBSD project: [freebsd-git](https://lists.freebsd.org/mailman/listinfo/freebsd-git) - -Commit messages will be sent to following mailing lists: - -- [dev-commits-doc-all](https://lists.freebsd.org/mailman/listinfo/dev-commits-doc-all): All changes to the doc repository -- [dev-commits-ports-all](https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-all): All changes to the ports repository -- [dev-commits-ports-main](https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-main): All changes to the "main" branch of the ports repository -- [dev-commits-ports-branches](https://lists.freebsd.org/mailman/listinfo/dev-commits-ports-branches): All changes to the quarterly branches of the ports repository -- [dev-commits-src-all](https://lists.freebsd.org/mailman/listinfo/dev-commits-src-all): All changes to the src repository -- [dev-commits-src-main](https://lists.freebsd.org/mailman/listinfo/dev-commits-src-main): All changes to the "main" branch of the src repository (the FreeBSD-CURRENT branch) -- [dev-commits-src-branches](https://lists.freebsd.org/mailman/listinfo/dev-commits-src-branches): All changes to all stable branches of the src repository - -For more information, please refer to the "Commit message lists" section of C.2. "Mailing Lists" in handbook: https://www.freebsd.org/doc/en/books/handbook/eresources-mail.html diff --git a/github-mirroring.md b/github-mirroring.md deleted file mode 100644 index 209223b..0000000 --- a/github-mirroring.md +++ /dev/null @@ -1,130 +0,0 @@ -# Github mirroring plan - -# How to migrate from old hashes to new - -Once we start mirroring the new git repo to github, old users of -https://github.com/freebsd/freebsd will need to update. - -## Simple case: just tracking - -If you are just tracking, without any changes, then there are only two cases. If you are tracking 'master' then you'll need to checkout main now: -``` -% git pull -% git checkout main -``` -If you are tracking one of the stable/XX branches, then a simple `git pull` will do the trick. - -## I have work off the `master` branch - -If you have branches that you need to migrate, there are two ways: in -place or separate tree. Most people will want to use the in-place -migration technique if they have only a few changes. For people with a -lot of changes, it may make sense to create a new tree and pull those -work branches in one at a time. - -### In Place - -Here the instructions are simple. Pull the new repo. For each branch you have, -rebase: -``` -% git rebase master my-branch --onto main -``` -this will rebase things forward and then you can continue as you have. - -### Separate tree - -If you have a lot of changes, it may make sense to use a fresh -tree. Clone the github repo. I'll assume that the two trees are -/tree/old and /tree/new for simplicity. - -``` -% cd /tree -% git clone https://github.com/freebsd/freebsd-src new -% cd new -% git remote add old /tree/old -``` -This creates the new tree and creates a remote called 'old'. Now, you can -pull in the branches you want to migrate, as you want to migrate them: -``` -% git fetch old my-branch -% git rebase master my-branch --onto main -``` - -This works becuase 'master' is the same in both repos. If you are -reading this after the planned removal of 'master' from the github -mirror, please see the section on migrating stable/X branches. - -### Using patches - -There's a third way if the repos are completely disconnected for -some reason. It's rather long, but see -http://bsdimp.blogspot.com/2020/08/how-to-transport-branch-from-one-git.html - -## Migrating a stable/XX branch - -Since we replaced the stable/XX definitions on github, if you have -branches from stable/XX you may need the old references. - -``` -% git remote add legacy https://github.com/freebsd/freebsd-legacy -% git fetch legacy -``` -This looks like it will fetch a buch of stuff, but really it just fetches the -legacy branch names so you can refer to them in the rebase: -``` -% git rebase legacy/stable/12 my-branch --onto stable/12 -``` -then is how to convert (once you've updated the stable/12 -branch from the old refs to the new). - -# Current Plans - -## FreeBSD History - -The FreeBSD project is currently mirroring to github an experimental -tree. Due to the large number of bugs that adversely affects the -quality of the tree, and its day to day use for anything more -complicated that merging and commits and simple history browsing, the -move to git project opted to redo the hashes and fix many historical -mistakes as part of its effort. As such, a migration plan is needed, -so this describes it. - -## The Plan for src - -1. Push a copy of the long-runing 'beta hashes' repo to github as freebsd-legacy -3. rename github/freebsd -> github/freebsd-src. Old users that have freebsd/freebsd - in their .git/config will get the new stream. -4. Add the merge commit to freebsd-legacy that we need for subtree merges (for each of - the branches that are active) -5. At some point, delete 'master' from the new freebsd-src. - -With this plan, all the upward migration paths are preserved. - -For folks tracking master, they will see no updates until they cut -over. This is -current, and people are expected to pay attention, so -this is OK. - -Folks with work off stable/* will see forced push and will have to do -something special. My notion is the special thing will be -``` -% git remote add legacy https://github.com/freebsd/freebsd-legacy -% git fetch legacy -% foreach branch B: - git rebase legacy/stable/X B --onto stable/X -``` -the fetch legacy will largely be a nop since they have all the -objects, just not the refs. Once they do the conversion, they can get -rid of the double history with git gc. - -We put this in the readme.md (and delete readme, but that's a separate -issue) and a pointer to the doc. - -## Plan for doc - -1. fork freebsd-doc-legacy -2. delete the delgacy stuff except master. -3. keep master around for N months (6?) and then delete it. -4. Add note to readme about how to jump. -5. NOTE: historical branches will be deleted so only 'main' reamins on github. - - From 966a25baac9d3337830889ea6f6d141f3009f286 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Tue, 16 Mar 2021 10:28:17 -0600 Subject: [PATCH 40/44] Vendor imports moved into handbook --- SUMMARY.adoc | 2 +- vendor.md | 204 --------------------------------------------------- 2 files changed, 1 insertion(+), 205 deletions(-) delete mode 100644 vendor.md diff --git a/SUMMARY.adoc b/SUMMARY.adoc index 4a9bea5..1a902ce 100644 --- a/SUMMARY.adoc +++ b/SUMMARY.adoc @@ -26,7 +26,7 @@ == `src` Specific * link:src-cvt.md[FreeBSD Src Committer Transition Guide] -* link:vendor.md[(WIP) How to do vendor imports] +* https://docs.freebsd.org/en/articles/committers-guide/#git-primer[How to do vendor imports] * link:MFC.md[(WIP) How to MFC] * link:projects-user.md[A brief note about Subversion Projects and User branches] diff --git a/vendor.md b/vendor.md deleted file mode 100644 index 5beb0be..0000000 --- a/vendor.md +++ /dev/null @@ -1,204 +0,0 @@ -# How to do vendor imports - -**THIS IS A DRAFT DOCUMENT AND CONTAINS SOME WORKING NOTES** - -With the FreeBSD Subversion to Git conversion complete, it's time to -look at how to do a vendor import. - -Note: This document follows the convention that the `freebsd` origin -is the source of truth. If you use a different convention, replace -freebsd with your name. - -To import notes (helpful when you want to look up svn revisions), for -example, one would clone the repository and then do: -``` -git config --add remote.freebsd.fetch '+refs/notes/*:refs/notes/*' && git fetch -``` - -All vendor branches and tags start with `vendor/` and are branches and -tags visible by default. Prior drafts of this document had them directly under -`refs/vendor` rather than `refs/heads/vendor` and `refs/tags/vendor` (the git -defaults) so early adopters that have the older refs may need to clean them up. - -We'll explore an example for updating NetBSD's mtree that's in our -tree. The vendor branch for this is `vendor/NetBSD/mtree`. - -## Updating an old vendor import - -Since the trees we have in vendor branches are usually a tiny subset of -the FreeBSD, it's best to do them with work trees since the process is -quite fast. Make sure that whatever directory you choose (the -`../mtree`) argument is empty and doesn't conflict. -``` -% git worktree add ../mtree vendor/NetBSD/mtree -``` -### Update the Sources in the Vendor Branch - -I have my copy of NetBSD checked out from their GitHub mirror in -`~/git/NetBSD`, so I'll update from there: Note that "upstream" might -have added or removed files, so we want to make sure deletions are -propagated as well. rsync(1) is commonly installed, so I'll use that. -``` -% cd ../mtree -% rsync -va --del --exclude=".git" ~/git/NetBSD/usr.sbin/mtree/ . -% git add -A -% git status -... -% git diff --staged -... -% git commit -m"Vendor import of NetBSD's mtree at 2020-12-11" -[vendor/NetBSD/mtree 8e7aa25fcf1] Vendor import of NetBSD's mtree at 2020-12-11 - 7 files changed, 114 insertions(+), 82 deletions(-) -% git tag -a vendor/NetBSD/mtree/20201211 -``` - -Note: I run the `git diff` and `git status` commands to make sure nothing weird -was present. Also I used `-m` to illustrate, but you should compose a proper -message in an editor (using a commit message template). - -It's also important to create an annotated tag, otherwise the push -will be rejected. Only annotated tags are allowed to be pushed. - -### Updating the FreeBSD Copy -At this point you can push the import to vendor into our repo. -``` -% git push --follow-tags freebsd vendor/NetBSD/mtree -``` - -`--follow-tags` tells `git push` to also push tags associated with the locally committed revision. - -### Updating the FreeBSD source tree -Now you need to update the mtree in FreeBSD. The sources live in -`contrib/mtree` since it is upstream software. - -``` -% cd ../src -% git subtree merge -P contrib/mtree vendor/NetBSD/mtree -``` -This would generate a subtree merge commit of `contrib/mtree` against the local `vendor/NetBSD/mtree` branch. -If there were conflicts, you would need to fix them before committing. - -### Rebasing your change against latest FreeBSD source tree - -Because the current policy recommends against using merges, if the upstream FreeBSD `main` moved forward -before you get a chance to push, you would have to redo the merge. - -Regular `git rebase` or `git pull --rebase` doesn't know how to rebase a merge commit **as a merge commit**, -so instead of that you would have to recreate the commit. - -The easiest way to do this would be to create a side branch with the **contents** of the merged tree: - -``` -% cd ../src -% git fetch freebsd -% git checkout -b merge_result -% git merge freebsd/main -``` - -Typically, there would be no merge conflicts here (because developers tends to work on different components). -In the worst case scenario, you would still have to resolve merge conflicts, if there was any, but this -should be really rare. - -Now, checkout `freebsd/main` again as `new_merge`, and redo the merge: - -``` -% git checkout -b new_merge freebsd/main -% git subtree merge -P contrib/mtree vendor/NetBSD/mtree -``` - -Instead of resolving the conflicts, perform this instead: - -``` -% git checkout merge_result . -``` - -Which will overwrite the files with conflicts with the version found in `merge_result`. - -Examine the tree against `merge_result` to make sure that you haven't missed deleted files: - -``` -% git diff merge_result -``` - -## Creating a new vendor branch -There's a number of ways to create a new vendor branch. The easiest is -to create a new repository and then merge that with FreeBSD. Let's say -we're importing `glorbnitz` into the FreeBSD tree, release 3.1415. For -the sake of simplicity, we'll not trim this release. It's a user -command that puts the nitz device into different magical glorb states. - -### Create the repo -``` -% cd /some/where -% mkdir glorbnitz -% cd glorbnitz -% git init -% git checkout -b vendor/glorbnitz -``` - -At this point, you have a new repo, where all new commits will go on -the `vendor/glorbnitz` branch. - -(Git professionals can also do this right in their FreeBSD clone, if they know -how to create a new root commit that's not attached to anything, e.g. -`git checkout --orphan vendor/glorbnitz`) - -### Copy the sources in -Since this is a new import, you can just cp the sources in, or use tar or -even rsync as shown above. And we'll add everything, assuming no dot files. -``` -% cp -r ~/glorbnitz/* . -% git add * -``` - -At this point, you should have a pristine copy of glorbnitz ready to commit. - -``` -% git commit -m"Import GlorbNitz frobnosticator revision 3.1415" -``` -As above, I used `-m` for simplicity, but you should likely create a -commit message that explains what a Glorb is and why you'd use a Nitz -to get it. Not everybody will know. - -### Now import it into our repository -Now you need to import the branch into our repository. -``` -% cd /path/to/freebsd/repo/src -% git remote add glorbnitz /some/where/glorbnitz -% git fetch glorbnitz vendor/glorbnitz -``` -Note the vendor/glorbnitz branch is in the repo. At this point the -`/some/where/glorbnitz` can be deleted, if you like. It was only a means -to an end. - -### Tag and push -Steps from here on out are much the same as they are in the case of -updating a vendor branch, though w/o the updating the vendor -branch step. -``` -% git worktree add ../glorbnitz vendor/glorbnitz -% cd ../glorbnitz -% git tag --annotate vendor/glorbnitz/3.1415 -# Make sure it's good -% git push --follow-tags freebsd vendor/glorbnitz -``` -By 'good' we mean: -1. All the right files are present -2. None of the wrong files are present -3. The vendor branch points at something sensible -4. The tag looks good, and is annotated. - -### Time to finally merge it into the base tree -``` -% cd ../src -% git subtree add -P contrib/glorbnitz vendor/glorbnitz -# Make sure it's good -% git push freebsd -``` -Here 'good' means: -1. All the right files, and none of the wrong ones, were merged into contrib/glorbnitz. -2. No other changes are in the tree -3. The commit messages look good. - -Note: This hasn't connected `glorbnitz` to the build yet. How to do -that hasn't changed and is beyond the scope of this article. From 6fc255f01976dc8ee0c8168a65f503fdf8efb1cc Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Tue, 16 Mar 2021 22:25:08 -0600 Subject: [PATCH 41/44] Update for stuff I've pushed into the tree --- MFC.md | 498 --------------------------------------------------- SUMMARY.adoc | 4 +- src-cvt.md | 252 -------------------------- 3 files changed, 2 insertions(+), 752 deletions(-) delete mode 100644 MFC.md delete mode 100644 src-cvt.md diff --git a/MFC.md b/MFC.md deleted file mode 100644 index 657b435..0000000 --- a/MFC.md +++ /dev/null @@ -1,498 +0,0 @@ -# How to MFC - -** NOTE: THIS IS WORK IN PROGRESS and current incomplete ** - -Note: This document uses the convention where the upstream origin name -is `freebsd` as suggested in other docs. - -## Summary - -MFC workflow can be summarized as `git cherry-pick -x` plus git commit ---amend to adjust the commit message. For multiple commits, use `git rebase -i` -to squash them together and edit the commit message. - -## Single commit MFC - -``` -% git checkout stable/X -% git cherry-pick -x $HASH --edit -``` - -For MFC commits, for example a vendor import, you would need to specify one parent for cherry-pick -purposes. Normally, that would be the "first parent" of the branch you are cherry-picking from, so: - -``` -% git checkout stable/X -% git cherry-pick -x $HASH -m 1 --edit -``` - -If things go wrong, you'll either need to abort the cherry-pick with `git cherry-pick --abort` or fix it -up and do a `git cherry-pick --continue`. - -Once the cherry-pick is finished, push with `git push`. If you get an error due to losing the commit race, -use `git pull --rebase` and try to push again. - -## Multiple commit MFC - -``` -% git checkout -b tmp-branch stable/X -% for h in $HASH_LIST; do git cherry-pick -x $h; done -% git rebase -i stable/X -# mark each of the commits after the first as 'squash' -# edit the commit message to be sane -% git push freebsd HEAD:stable/X -``` - -If the push fails due to losing the commit race, rebase and try again: - -``` -% git checkout stable/X -% git pull -% git checkout tmp-branch -% git rebase stable/X -% git push freebsd HEAD:stable/X -``` - -Once the MFC is complete, you can delete the temporary branch: - -``` -% git checkout stable/X -% git branch -d tmp-branch -``` - -## MFC a vendor import - -Vendor imports are the only thing in the tree that creates a merge -commit in the main line. Cherry picking merge commits into stable/XX -presents an additional difficulty because there are two parents for a -merge commit. Generally, you'll want the first parent's diff since -that's the diff to mainline (though there may be some exceptions). - -``` -% git cherry-pick -x -m 1 $HASH -``` -is typically what you want. This will tell cherry-pick to apply the correct diff. - -There are some, hopefully, rare cases where it's possible that the -mainline was merged backwards by the conversion script. Should that be -the case (and we've not found any yet), you'd change the above to '-m 2' -to pickup the proper parent. Just do -``` -% git cherry-pick --abort -% git cherry-pick -x -m 2 $HASH -``` -to do that. The `--aboort` will cleanup the failed first attempt. - -## Redoing a MFC - -If you do a MFC, and it goes horribly wrong and you want to start over, -then the easiest way is to use `git reset --hard` like so: -``` -% git reset --hard freebsd/stable/12 -``` -though if you have some revs you want to keep, and others you don't, -using 'git rebase -i' is better. - -## Considerations when MFCing - -When committing source commits to stable and releng branches, we have -the following goals: - -1. Clearly mark direct commits distinct from commits that land a - change from another branch -2. Avoid introducing known breakage into stable and releng branches -3. Allow developers to determine which changes have or have not been - landed from one branch to another - -With subversion, we used the following practices to achieve these goals: - -1. Using 'MFC' and 'MFS' tags to mark commits that merged changes from - another branch -2. Squashing fixup commits into the main commit when merging a change -3. Recording mergeinfo so that `svn mergeinfo --show-revs` worked - -With Git, we will need to use different strategies to achieve the same -goals. This document aims to define best practices when merging -source commits using git that achieve these goals. In general, we aim -to use git's native support to achieve these goals rather than -enforcing practices built on subversion's model. - -One general note: due to technical differences with Git, we will not -be using git "merge commits" (created via `git merge`) in stable or -releng branches. Instead, when this document refers to "merge -commits", it means a commit originally made to `main` that is -replicated or "landed" to a stable branch, or a commit from a stable -branch that is replicated to a releng branch with some varation of -`git cherry-pick`. - -## Commit message standards - -### Marking MFCs - -There are two main options for marking MFCs as distinct from direct -commits: - -1. One option that matches our existing practice (the wisdom of which - I'm not commenting on) would mark MFCs like this in the commit - message: - -``` -MFC: 12def6789a3a,ac32ee4a5c2e -``` - - where the first 12 digits of the hash is used to mark the commit message. - This "abbreviated hash" can be retrieved by: - -``` -git show --format=%p --no-patch $full_hash -``` - - This preserves the information, but isn't 'git standard'. It also - requires committers to manually edit commit messages to include - this information when merging. - -2. Use the `-x` flag with `git cherry-pick`. This adds a line to the - commit message that includes the hash of the original commit when - merging. Since it is added by Git directly, committers do not have - to manually edit the commit log when merging. - -We feel that the second option is simpler going forward. - -### Finding Eligible Hashes to MFC - -Git provides some built-in support for this via the `git cherry` and -`git log --cherry` commands. These commands compare the raw diffs of -commits (but not other metadata such as log messages) to determine if -two commits are identical. This works well when each commit from head -is landed as a single commit to a stable branch, but it falls over if -multiple commits from main are squashed together as a single commit to -a stable branch. - -There are a few options for resolving this: - -1. We could ban squashing of commits and instead require that committers - stage all of the fixup / follow-up commits to stable into a single - push. This would still achieve the goal of stability in stable and - releng branches since pushes are atomic and users doing a simple pull - will never end up with a tree that has the main commit without the - fixup(s). `git bisect` is also able to cope with this model via - `git bisect skip`. - -2. We could adopt a consistent style for describing MFCs and write - our own tooling to wrap around `git cherry` to determine the list - of eligible commits. A simple approach here might be to use the - syntax from `git cherry-pick -x`, but require that a squashed - commit list all of the hashes (one line per hash) at the end of - the commit message. Developers could do this by using - `git cherry-pick -x` of each individual commit into a branch and - then use `git rebase` to squash the commits down into a single - commit, but collecting the `-x` annotations at the end of the - landed commit log. - -### Trim Metadata? - -One area that was not clearly documented with subversion (or even CVS) -is how to format metadata in log messages for MFC commits. Should -it include the metadata from the original commit unchanged, or should -it be altered to reflect information about the MFC commit itself? - -Historical practice has varied, though some of the variance is by -field. For example, MFCs that are relevant to a PR generally -include the PR field in the MFC so that MFC commits are included -in the bug tracker's audit trail. Other fields are less clear. For -example, Phabricator shows the diff of the last commit tagged to a -review, so including Phabricator URLs replaces the `main` commit with -the landed commits. The list of reviewers is also not clear. If a -reviewer has approved a change to `main`, does that mean they have -approved the MFC commit? Is that true if it's identical code only, -or with merely trivial reworkes? It's clearly not true for more -extensive reworks. Even for identical code what if the commit doesn't -conflict but introduces an ABI change? A reviewer may have ok'd a -commit for `main` due to the ABI breakage but may not approve of -merging the same commit as-is. One will have to use one's best -judgement until clear guidelines can be agreed upon. - -For MFCs regulated by re@, new metadata fields are added, such as -the Approved by tag for approved commits. This new metadata will have -to be added via `git commit --amend` or similar after the original -commit has been reviewed and approved. We may also want to reserve -some metadata fields in MFC commits such as Phabricator URLs for use -by re@ in the future. - -Preserving existing metadata provides a very simple workflow. -Developers can just use `git cherry-pick -x` without having to edit -the log message. - -If instead we choose to adjust metadata in MFCs, developers will -have to edit log messages explicitly via the use of `git cherry-pick ---edit` or `git commit --amend`. However, as compared to svn, at -least the existing commit message can be pre-populated and metadata -fields can be added or removed without having to re-enter the entire -commit message. - -The bottom line is that developers will likely need to curate their -commit message for MFCs that are non-trivial. - -## Looking for things to MFC - -If you are looking for changes to MFC, the following may help: -``` -% git log --cherry stable/12 main -- bin/ls -``` - -## Scripts - -We currently have no scripts. - -## Examples - -### Merging a Single Subversion Commit - -This walks through the process of merging a commit to stable/12 that -was originally committed to head in Subversion. In this case, the -original commit is r368685. - -The first step is to map the Subversion commit to a Git hash. Once -you have fetched refs/notes/commits, you can pass the revision number -to `git log --grep`: - -``` -> git log main --grep 368685 -commit ce8395ecfda2c8e332a2adf9a9432c2e7f35ea81 -Author: John Baldwin -Date: Wed Dec 16 00:11:30 2020 +0000 - - Use the 't' modifier to print a ptrdiff_t. - - Reviewed by: imp - Obtained from: CheriBSD - Sponsored by: DARPA - Differential Revision: https://reviews.freebsd.org/D27576 - -Notes: - svn path=/head/; revision=368685 -``` - -Next, MFC the commit to a `stable/12` checkout: - -``` -git checkout stable/12 -git cherry-pick -x ce8395ecfda2c8e332a2adf9a9432c2e7f35ea81 --edit -``` - -Git will invoke the editor. Use this to remove the metadata that only -applied to the original commit (Phabricator URL and Reviewed by). -After the editor saves the updated log message, Git completes the -commit: - -``` -[stable/12 3e3a548c4874] Use the 't' modifier to print a ptrdiff_t. - Date: Wed Dec 16 00:11:30 2020 +0000 - 1 file changed, 1 insertion(+), 1 deletion(-) -``` - -The contents of the MFCd commit can be examined via `git show`: - -``` -> git show -commit 3e3a548c487450825679e4bd63d8d1a67fd8bd2d (HEAD -> stable/12) -Author: John Baldwin -Date: Wed Dec 16 00:11:30 2020 +0000 - - Use the 't' modifier to print a ptrdiff_t. - - Obtained from: CheriBSD - Sponsored by: DARPA - - (cherry picked from commit ce8395ecfda2c8e332a2adf9a9432c2e7f35ea81) - -diff --git a/sys/compat/linuxkpi/common/include/linux/printk.h b/sys/compat/linuxkpi/common/include/linux/printk.h -index 31802bdd2c99..e6510e9e9834 100644 ---- a/sys/compat/linuxkpi/common/include/linux/printk.h -+++ b/sys/compat/linuxkpi/common/include/linux/printk.h -@@ -68,7 +68,7 @@ print_hex_dump(const char *level, const char *prefix_str, - printf("[%p] ", buf); - break; - case DUMP_PREFIX_OFFSET: -- printf("[%p] ", (const char *)((const char *)buf - -+ printf("[%#tx] ", ((const char *)buf - - (const char *)buf_old)); - break; - default: -``` - -The MFC commit can now be published via `git push` - -``` -git push freebsd -Enumerating objects: 17, done. -Counting objects: 100% (17/17), done. -Delta compression using up to 4 threads -Compressing objects: 100% (7/7), done. -Writing objects: 100% (9/9), 817 bytes | 204.00 KiB/s, done. -Total 9 (delta 5), reused 1 (delta 1), pack-reused 0 -To gitrepo-dev.FreeBSD.org:src.git - 525bd9c9dda7..3e3a548c4874 stable/12 -> stable/12 -``` - -### Merging a Single Subversion Commit with a Conflict - -This example is similar to the previous example except that the -commit in question encounters a merge conflict. In this case, the -original commit is r368314. - -As above, the first step is to map the Subversion commit to a Git -hash: - -``` -> git log main --grep 368314 -commit 99963f5343a017e934e4d8ea2371a86789a46ff9 -Author: John Baldwin -Date: Thu Dec 3 22:01:13 2020 +0000 - - Don't transmit mbufs that aren't yet ready on TOE sockets. - - This includes mbufs waiting for data from sendfile() I/O requests, or - mbufs awaiting encryption for KTLS. - - Reviewed by: np - MFC after: 2 weeks - Sponsored by: Chelsio Communications - Differential Revision: https://reviews.freebsd.org/D27469 - -Notes: - svn path=/head/; revision=368314 -``` - -Next, MFC the commit to a `stable/12` checkout: - -``` -git checkout stable/12 -git cherry-pick -x 99963f5343a017e934e4d8ea2371a86789a46ff9 --edit -Auto-merging sys/dev/cxgbe/tom/t4_cpl_io.c -CONFLICT (content): Merge conflict in sys/dev/cxgbe/tom/t4_cpl_io.c -warning: inexact rename detection was skipped due to too many files. -warning: you may want to set your merge.renamelimit variable to at least 7123 and retry the command. -error: could not apply 99963f5343a0... Don't transmit mbufs that aren't yet ready on TOE sockets. -hint: after resolving the conflicts, mark the corrected paths -hint: with 'git add ' or 'git rm ' -hint: and commit the result with 'git commit' -``` - -In this case, the commit encountered a merge conflict in -sys/dev/cxge/tom/t4_cpl_io.c as kernel TLS is not present in -stable/12. Note that Git does not invoke an editor to adjust the -commit message due to the conflict. `git status` confirms that this -file has merge conflicts: - -``` -> git status -On branch stable/12 -Your branch is up to date with 'upstream/stable/12'. - -You are currently cherry-picking commit 99963f5343a0. - (fix conflicts and run "git cherry-pick --continue") - (use "git cherry-pick --skip" to skip this patch) - (use "git cherry-pick --abort" to cancel the cherry-pick operation) - -Unmerged paths: - (use "git add ..." to mark resolution) - both modified: sys/dev/cxgbe/tom/t4_cpl_io.c - -no changes added to commit (use "git add" and/or "git commit -a") -``` - -After editing the file to resolve the conflict, `git status` shows the -conflict as resolved: - -``` -> git status -On branch stable/12 -Your branch is up to date with 'upstream/stable/12'. - -You are currently cherry-picking commit 99963f5343a0. - (all conflicts fixed: run "git cherry-pick --continue") - (use "git cherry-pick --skip" to skip this patch) - (use "git cherry-pick --abort" to cancel the cherry-pick operation) - -Changes to be committed: - modified: sys/dev/cxgbe/tom/t4_cpl_io.c -``` - -The cherry-pick can now be completed: - -``` -> git cherry-pick --continue -``` - -Since there was a merge conflict, Git invokes the editor to -adjust the commit message. Trim the metadata fields from the -commit log from the original commit to head and save the -updated log message. - -The contents of the MFC commit can be examined via `git show`: - -``` -> git show -commit 525bd9c9dda7e7c7efad2d4570c7fd8e1a8ffabc (HEAD -> stable/12) -Author: John Baldwin -Date: Thu Dec 3 22:01:13 2020 +0000 - - Don't transmit mbufs that aren't yet ready on TOE sockets. - - This includes mbufs waiting for data from sendfile() I/O requests, or - mbufs awaiting encryption for KTLS. - - Sponsored by: Chelsio Communications - - (cherry picked from commit 99963f5343a017e934e4d8ea2371a86789a46ff9) - -diff --git a/sys/dev/cxgbe/tom/t4_cpl_io.c b/sys/dev/cxgbe/tom/t4_cpl_io.c -index 8e8c2b8639e6..43861f10b689 100644 ---- a/sys/dev/cxgbe/tom/t4_cpl_io.c -+++ b/sys/dev/cxgbe/tom/t4_cpl_io.c -@@ -746,6 +746,8 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) - for (m = sndptr; m != NULL; m = m->m_next) { - int n; - -+ if ((m->m_flags & M_NOTAVAIL) != 0) -+ break; - if (IS_AIOTX_MBUF(m)) - n = sglist_count_vmpages(aiotx_mbuf_pages(m), - aiotx_mbuf_pgoff(m), m->m_len); -@@ -821,8 +823,9 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) - - /* nothing to send */ - if (plen == 0) { -- KASSERT(m == NULL, -- ("%s: nothing to send, but m != NULL", __func__)); -+ KASSERT(m == NULL || (m->m_flags & M_NOTAVAIL) != 0, -+ ("%s: nothing to send, but m != NULL is ready", -+ __func__)); - break; - } - -@@ -910,7 +913,7 @@ t4_push_frames(struct adapter *sc, struct toepcb *toep, int drop) - toep->txsd_avail--; - - t4_l2t_send(sc, wr, toep->l2te); -- } while (m != NULL); -+ } while (m != NULL && (m->m_flags & M_NOTAVAIL) == 0); - - /* Send a FIN if requested, but only if there's no more data to send */ - if (m == NULL && toep->flags & TPF_SEND_FIN) -``` - -The MFC commit can now be published via `git push` - -``` -git push freebsd -Enumerating objects: 13, done. -Counting objects: 100% (13/13), done. -Delta compression using up to 4 threads -Compressing objects: 100% (7/7), done. -Writing objects: 100% (7/7), 819 bytes | 117.00 KiB/s, done. -Total 7 (delta 6), reused 0 (delta 0), pack-reused 0 -To gitrepo.FreeBSD.org:src.git - f4d0bc6aa6b9..525bd9c9dda7 stable/12 -> stable/12 -``` diff --git a/SUMMARY.adoc b/SUMMARY.adoc index 1a902ce..dfe1475 100644 --- a/SUMMARY.adoc +++ b/SUMMARY.adoc @@ -25,9 +25,9 @@ == `src` Specific -* link:src-cvt.md[FreeBSD Src Committer Transition Guide] +* https://docs.freebsd.org/en/articles/committers-guide/#git-primer[FreeBSD Src Committer Transition Guide] * https://docs.freebsd.org/en/articles/committers-guide/#git-primer[How to do vendor imports] -* link:MFC.md[(WIP) How to MFC] +* https://docs.freebsd.org/en/articles/committers-guide/#git-primer[How to MFC] * link:projects-user.md[A brief note about Subversion Projects and User branches] == Git References diff --git a/src-cvt.md b/src-cvt.md deleted file mode 100644 index 3075a94..0000000 --- a/src-cvt.md +++ /dev/null @@ -1,252 +0,0 @@ -# FreeBSD Src Committer Transition Guide - -This document is designed to walk people through the conversion -process from Subversion to Git, written from the source committer's point -of view. This document is a living document, so please don't hesitate -to send improvements, or even ask for areas to be explained more / -better / at all. - -## Old vs New URL translation table - -Before we get started, here's a handy cheat sheet for old to new URLs. - -SVN infra -> Git infra map - -| Item | SVN | Git | -| ---------------------------------------- | ------------------------------- | ----------------------------------- | -| Web-based repository browser | https://svnweb.freebsd.org | https://cgit.freebsd.org | -| Distributed mirrors for anonymous readonly checkout/clone | https://svn.freebsd.org svn://svn.freebsd.org | https://git.freebsd.org ssh://anongit@git.freebsd.org | -| Read/write Repository for committers (*) | svn+ssh://(svn)repo.freebsd.org | ssh://git@(git)repo.freebsd.org | - -(*) Before all repositories in SVN have been migrated, the repo.freebsd.org will be pointing to one of: - - svnrepo.freebsd.org - - gitrepo.freebsd.org - -Please use the hostname that explicitly includes the VCS name to -access the right repositories during the migration. `repo.freebsd.org` -will be the canonical FreeBSD Git repository for the committers after -all the repositories migrated to Git. - -## Git basics - -There are many primers on how to use Git on the web. There's a lot of -them (google "Git primer"). This one comes up first, and is generally -good. https://danielmiessler.com/study/git/ and -https://gist.github.com/williewillus/068e9a8543de3a7ef80adb2938657b6b -are good overviews. The Git book is also complete, but much longer -https://git-scm.com/book/en/v2. There is also this website -https://ohshitgit.com/ for common traps and pitfalls of Git, in case -you need guidance to fix things up. - -This document will assume that you've read through it and will try not -to belabor the basics (though it will cover them briefly). - -## Migrating from a Subversion tree - -This section will cover a couple of common scenarios for migrating -from using the FreeBSD Subversion repo to the FreeBSD source git repo. The -FreeBSD Git conversion is still in beta status, so some minor things -may change between this and going into production. - -Before you git started, you'll need a copy of Git. Any Git will do, -though the latest ones are always recommended. Either build it from -ports, or install it using pkg (though some folks might use `su` or -`doas` instead of `sudo`): -``` -% sudo pkg install git -``` - -### No staged changes migration - -If you have no changes pending, the migration is straight forward. In -this, you abandon the Subversion tree and clone the Git repo. It's -likely best to retain your subversion tree, in case there's something -you've forgotten about there. First, let's clone a repo: -``` -% git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/src.git freebsd-src -``` -will create a clone of the FreeBSD src repo into a subdirectory called -`freebsd-src` and include the 'notes' about the revisions. -The current plan for GitHub mirroring is to mirror to -https://github.com/freebsd/freebsd.git as well. When the transition -starts, the github `master` branch will be frozen. We will be using the name `main` instead -of `master` that was used in the beta version of the github.com mirror. -The exact logistics of this are still being finalized, as there are over 2k forks and 5k stars. -We will also mirror the repo to gitlab at https://gitlab.com/FreeBSD/src.git . -Its transition plan is also being finalized. - -It's useful to have the old Subversion revisions available. This data is stored -using Git notes, but Git doesn't fetch those by default. The --config -and the argument above changed the default to fetch the notes. If -you've cloned the repo without this, or wish to add notes to an -previously clone repository, use the following commands: -``` -% git config --add remote.freebsd.fetch "+refs/notes/*:refs/notes/*" -% git fetch -``` -At this point you have the src checked out into a Git tree, ready to -do other things. - -### But I have changes that I've not committed - -If you are migrating from a tree that has changes you've not yet -committed to FreeBSD, you'll need to follow the steps from the -previous section first, and then follow these. -``` -% cd path-to-svn-checkout-tree -% svn diff > /tmp/src.diff -% cd mumble/freebsd-src -% git checkout -b working -``` -This will create a diff of your current changes. The last command -creates a branch called `working` though you can call it whatever you -want. -``` -% git apply /tmp/src.diff -``` -this will apply all your pending changes to the working tree. This -doesn't commit the change, so you'll need to make this permanent: -``` -% git commit -``` -The last command will commit these changes to the branch. The editor -will prompt you for a commit message. Enter one as if you were -committing to FreeBSD. - -At this point, your work is preserved, and in the Git repo. - -### Keeping current - -So, time passes. It's time now to update the tree for the latest -changes upstream. When you checkout `main` make sure that you have no -diffs. It's a lot easier to commit those to a branch (or use `git -stash`) before doing the following. - -If you are used to `git pull`, I would strongly recommend using the -`--ff-only` option, and further setting it as the default option. -Alternatively, `git pull --rebase` is useful if you have changes staged -in the main directory. -``` -% git config --global pull.ff only -``` -``` -% cd freebsd-src -% git checkout main -% git pull (--ff-only|--rebase) -``` -There is a common trap, that the combination command `git pull` will -try to perform a merge, which would sometimes creates a merge commit -sha that didn't exist before. This can be harder to recover from. - -The longer form is also recommended. -``` -% cd freebsd-src -% git checkout main -% git fetch freebsd -% git merge --ff-only freebsd/main -``` -These commands reset your tree to the main branch, and then update it -from where you pulled the tree from originally. It's important to -switch to `main` before doing this so it moves forward. Now, it's time -to move the changes forward: -``` -% git rebase -i main working -``` -This will bring up an interactive screen to change the defaults. For -now, just exit the editor. Everything should just apply. If not, then -you'll need to resolve the -diffs. https://docs.github.com/en/free-pro-team@latest/github/using-git/resolving-merge-conflicts-after-a-git-rebase -can help you navigate this process. - -### Time to push changes upstream - -First, ensure that the push URL is properly configured for the upstream -repository. -``` -% git remote set-url --push freebsd ssh://git@gitrepo.freebsd.org/src.git -``` -Then, verify that user name and email are configured right. We require -that they exactly match the passwd entry in FreeBSD cluster. Use -``` -freefall% gen-gitconfig.sh -``` -on freefall.freebsd.org to get recipe that you can use directly, assuming -/usr/local/bin is in the PATH. - -The below command merges the 'working' branch into the upstream main line. -It's important that you curate your changes to be just -like you want them in the FreeBSD source repo before doing this. -``` -% git push freebsd working:main -``` - -If your push is rejected due to losing a commit race, rebase your branch -before trying again: -``` -% git checkout working -% git fetch freebsd -% git rebase freebsd/main -% git push freebsd working:main -``` - -### Finding the Subversion Revision - -You'll need to make sure that you've fetched the notes (see the `No -staged changes migration` section above for details. Once you have -these, notes will show up in the git log command like so: -``` -% git log -XXX NEED to UPDATE -``` - -If you have a specific version in mind, you can use this construct: -``` -% git log --grep revision=XXXX -XXX need to update -% -``` -to find the specific revision. The hex number after 'commit' is the -hash you can use to refer to this commit. - -## Migrating from GitHub fork - -Note: as of this writing, the https://github.com/freebsd/freebsd-src -is mirroring all official branches, along with a `master` branch which -is the leagcy svn2git result. The `master` branch will not be updated anymore, -and the [last commit](https://github.com/freebsd/freebsd-src/commit/de1aa3dab23c06fec962a14da3e7b4755c5880cf) -contains the instructions of migrating to new `main` branch. -We'll likely retain the `master` branch for a certain time, but in the future -it will only be kept in the -[freebsd-legacy](https://github.com/freebsd/freebsd-legacy) repository. - -When migrating branches from a github fork from the old github mirror -to the official repo, the process is straight forward. This assumes that -you have a `freebsd` upstream pointing to github, adjust if necessary. -This also assumes a clean tree before starting... -1. Add the new `freebsd` source of truth: -``` -% git remote add freebsd https://git.freebsd.org/src.git -% git fetch freebsd -% git checkout --track freebsd/main -``` -2. Rebase all your WIP branches. For each branch FOO, do the following after -fetching the `freebsd` sources and creating a local `main` reference with -the above checkout: -``` -% git rebase -i freebsd/master FOO --onto main -``` -And you'll now be tracking the official source of truth. You can then follow -the `Keeping Current` section above to stay up to date. - -If you need to then commit work to FreeBSD, you can do so following the -`Time to push changes upstream` instructions. You'll need to do the following -once to update the push URL if you are a FreeBSD committer: -``` -% git remote set-url --push freebsd ssh://git@gitrepo.freebsd.org/src.git -(note that gitrepo.freebsd.org will be change to repo.freebsd.org in the future.) -``` -You will also need to add `freebsd` as the location to push to. The -author recommends that your upstream github repo remain the default -push location so that you only push things into FreeBSD you intend to -by making it explicit. - From 41ad899d9c840d72a2c010d3ed9f5ef900cffca9 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Wed, 17 Mar 2021 00:58:53 -0600 Subject: [PATCH 42/44] Updates for what I've pushed into the tree --- SUMMARY.adoc | 12 +- commit.md | 2 +- doc-cvt.md | 257 ---------------------------- mini-primer.md | 455 ------------------------------------------------- 4 files changed, 3 insertions(+), 723 deletions(-) delete mode 100644 doc-cvt.md delete mode 100644 mini-primer.md diff --git a/SUMMARY.adoc b/SUMMARY.adoc index dfe1475..ab2734c 100644 --- a/SUMMARY.adoc +++ b/SUMMARY.adoc @@ -16,15 +16,8 @@ == Getting Started * link:git-how.md[How FreeBSD implements Git] -* link:mini-primer.md[FreeBSD mini-git Primer] +* https://docs.freebsd.org/en/articles/committers-guide/#git-mini-primer[FreeBSD mini-git Primer] * https://docs.freebsd.org/en/books/handbook/mirrors/#git[Important URLs of the FreeBSD Git Infrastructure] - -== `doc` Specific - -* link:doc-cvt.md[FreeBSD Doc Committer Transition Guide] - -== `src` Specific - * https://docs.freebsd.org/en/articles/committers-guide/#git-primer[FreeBSD Src Committer Transition Guide] * https://docs.freebsd.org/en/articles/committers-guide/#git-primer[How to do vendor imports] * https://docs.freebsd.org/en/articles/committers-guide/#git-primer[How to MFC] @@ -36,8 +29,7 @@ == Committer References -* link:commit.md[Writing Good FreeBSD Commit Messages] -* link:meta.md[Include appropriate metadata in a footer] +* https://docs.freebsd.org/en/articles/committers-guide/#commit-log-message[Writing Good FreeBSD Commit Messages] * link:pull-request.md[How to deal with Pull Requests] == Design & Implementation diff --git a/commit.md b/commit.md index d0e119f..25f4a5b 100644 --- a/commit.md +++ b/commit.md @@ -1,3 +1,3 @@ # Writing Good FreeBSD Commit Messages -Now available in the [committer's guide](https://docs.freebsd.org/en/articles/committers-guide/#commit-log-message). +Now available in the diff --git a/doc-cvt.md b/doc-cvt.md deleted file mode 100644 index d5b4bff..0000000 --- a/doc-cvt.md +++ /dev/null @@ -1,257 +0,0 @@ -# FreeBSD Doc Committer Transition Guide - -This document is designed to walk people through the conversion -process from Subversion to Git, written from the doc committer's point -of view. This document is a living document, so please don't hesitate -to send improvements, or even ask for areas to be explained more / -better / at all. - -## Old vs New URL translation table - -Before we get started, here's a handy cheat sheet for old to new URLs. -For complete information, please see [URLs.md]. - -SVN infra -> Git infra map - -| Item | SVN | Git | -| ---------------------------------------- | ------------------------------- | ----------------------------------- | -| Web-based repository browser | https://svnweb.freebsd.org | https://cgit.freebsd.org | -| Distributed mirrors for anonymous readonly checkout/clone | https://svn.freebsd.org svn://svn.freebsd.org | https://git.freebsd.org/doc.git ssh://anongit@git.freebsd.org/doc.git | -| Read/write Repository for committers (*) | svn+ssh://(svn)repo.freebsd.org/doc | ssh://git@(git)repo.freebsd.org/doc.git | - -(*) Before all repositories in SVN have been migrated, the repo.freebsd.org will be pointing to one of: - - svnrepo.freebsd.org - - gitrepo.freebsd.org - -Please use the hostname that explicitly includes the VCS name to -access the right repositories during the migration. `repo.freebsd.org` -will be the canonical FreeBSD Git repository for the committers after -all the repositories migrated to Git. - -## Git basics - -There are many primers on how to use Git on the web. There's a lot of -them (google "Git primer"). This one comes up first, and is generally -good. https://danielmiessler.com/study/git/ and -https://gist.github.com/williewillus/068e9a8543de3a7ef80adb2938657b6b -are good overviews. The Git book is also complete, but much longer -https://git-scm.com/book/en/v2. There is also this website -https://ohshitgit.com/ for common traps and pitfalls of Git, in case -you need guidance to fix things up. - -This document will assume that you've read through it and will try not -to belabor the basics (though it will cover them briefly). - -## Migrating from a Subversion tree - -This section will cover a couple of common scenarios for migrating -from using the FreeBSD Subversion repo to the FreeBSD docs repo. The -FreeBSD Git conversion is still in beta status, so some minor things -may change between this and going into production. - -Before you git started, you'll need a copy of Git. Any Git will do, -though the latest ones are always recommended. Either build it from -ports, or install it using pkg (though some folks might use `su` or -`doas` instead of `sudo`): -``` -% sudo pkg install git -``` - -### No staged changes migration - -If you have no changes pending, the migration is straight forward. In -this, you abandon the Subversion tree and clone the Git repo. It's -likely best to retain your subversion tree, in case there's something -you've forgotten about there. First, let's clone a repo: -``` -% mkdir git-docs -% cd git-docs -% git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' https://git.freebsd.org/doc.git freebsd-doc -``` -will create a clone of the FreeBSD doc repo into a subdirectory called -`freebsd-doc` and include the 'notes' about the revisions. I selected -that name because it is also the name we publish at github. -The current plan for GitHub mirroring is to mirror to -https://github.com/freebsd/freebsd-doc.git as well, but it is currently -frozen as the point of the transition (notice that the main branch is -still "master" instead of "main") while some logistics are being worked out. -We will also mirror the repo to gitlab at https://gitlab.com/FreeBSD/doc.git . -Its transition plan is also being finalized. - -It's useful to have the old Subversion revisions. This data is stored -using Git notes, but Git doesn't fetch those by default. The --config -and the argument above changed the default to fetch the notes. If -you've cloned the repo without this, or wish to add notes to an -previously clone repository, use the following commands: -``` -% git config --add remote.freebsd.fetch "+refs/notes/*:refs/notes/*" -% git fetch freebsd -``` -At this point you have the docs checked out into a Git tree, ready to -do other things. - -### But I have changes that I've not committed - -If you are migrating from a tree that has changes you've not yet -committed to FreeBSD, you'll need to follow the steps from the -previous section first, and then follow these. -``` -% cd path-to-svn-checkout-tree -% svn diff > /tmp/docs.diff -% cd git-docs/freebsd-doc -% git checkout -b working -``` -This will create a diff of your current changes. The last command -creates a branch called `working` though you can call it whatever you -want. -``` -% git apply /tmp/docs.diff -``` -this will apply all your pending changes to the working tree. This -doesn't commit the change, so you'll need to make this permanent: -``` -% git commit -``` -The last command will commit these changes to the branch. The editor -will prompt you for a commit message. Enter one as if you were -committing to FreeBSD. - -At this point, your work is preserved, and in the Git repo. - -### Keeping current - -So, time passes. It's time now to update the tree for the latest -changes upstream. When you checkout `main` make sure that you have no -diffs. It's a lot easier to commit those to a branch (or use `git -stash`) before doing the following. - -If you are used to `git pull`, I would strongly recommend using the -`--ff-only` option, and further setting it as the default option. -``` -% git config --global pull.ff only -``` -``` -% cd freebsd-doc -% git checkout main -% git pull (--ff-only) -``` -There is a common trap, that the combination command `git pull` will -try to perform a merge, which would sometimes creates a merge commit -sha that didn't exist before. This can be harder to recover from. - -The longer form is also recommended. -``` -% cd freebsd-doc -% git checkout main -% git fetch freebsd -% git merge --ff-only freebsd/main -``` -These commands reset your tree to the main branch, and then update it -from where you pulled the tree from originally. It's important to -switch to `main` before doing this so it moves forward. Now, it's time -to move the changes forward: -``` -% git rebase -i main working -``` -This will bring up an interactive screen to change the defaults. For -now, just exit the editor. Everything should just apply. If not, then -you'll need to resolve the -diffs. https://docs.github.com/en/free-pro-team@latest/github/using-git/resolving-merge-conflicts-after-a-git-rebase -can help you navigate this process. - -### Time to push changes upstream - -The below command merges the 'working' branch into the upstream main line. -It's important that you curate your changes to be just -like you want them in the FreeBSD doc repo before doing this. -``` -% git push freebsd working:main -``` - -If your push is rejected due to losing a commit race, rebase your branch -before trying again: -``` -% git checkout working -% git fetch freebsd -% git rebase freebsd/main -% git push freebsd working:main -``` - -### Finding the Subversion Revision - -You'll need to make sure that you've fetched the notes (see the `No -staged changes migration` section above for details. Once you have -these, notes will show up in the git log command like so: -``` -commit 77788b7b18c1569e368e845cfbd4739329a2b76f -Author: phk -Date: Mon Nov 23 09:00:37 2020 +0000 - - Update my GPG key - -Notes: - svn path=/head/; revision=54705 -``` - -If you have a specific version in mind, you can use this construct: -``` -% git log --grep revision=54700 -commit 9a9960bfb7e8ff9fa7fe04eb488eabb1fd06814b -Author: dbaio -Date: Sun Nov 22 20:57:38 2020 +0000 - - pt_BR/articles/freebsd-releng: Sync with en_US r54689 - - Approved by: ebrandi (doc) - Obtained from: https://translate-dev.freebsd.org - Differential Revision: https://reviews.freebsd.org/D27315 - -Notes: - svn path=/head/; revision=54700 -% -``` -to find the specific revision. The hex number after 'commit' is the -hash you can use to refer to this commit. - -## Migrating from GitHub fork - -Note: as of this writing, the https://github.com/freebsd/freebsd-doc -is mirroring the official `main` branch, along with a `master` branch which -is the leagcy svn2git result. The `master` branch will not be updated anymore. -We'll likely retain the `master` branch for a certain time, but in the future -it will only be kept in the -[freebsd-doc-legacy](https://github.com/freebsd/freebsd-doc-legacy) repository. - -When migrating branches from a github fork from the old github mirror -to the official repo, the process is straight forward. This assumes that -you have a `origin` upstream pointing to github, adjust if necessary. -This also assumes a clean tree before starting... It's also recommended to -run `git gc` beforehand in order to speed up the process of adding a new -remote. -1. Add the new `freebsd` source of truth: -``` -% git remote add freebsd https://git.freebsd.org/doc.git -% git fetch freebsd -% git checkout freebsd/main -``` -2. Rebase all your WIP branches. For each branch FOO, do the following after -fetching the `freebsd` sources and creating a local `main` reference with -the above checkout: -``` -% git rebase -i origin/master FOO --onto main -``` -And you'll now be tracking the official source of truth. You can then follow -the `Keeping Current` section above to stay up to date. - -If you need to then commit work to FreeBSD, you can do so following the -`Time to push changes upstream` instructions. You'll need to do the following -once to update the push URL if you are a FreeBSD committer: -``` -% git remote set-url --push freebsd ssh://git@gitrepo.freebsd.org/doc.git -(note that gitrepo.freebsd.org will be change to repo.freebsd.org in the future.) -``` -You will also need to add `freebsd` as the location to push to. The -author recommends that your upstream github repo remain the default -push location so that you only push things into FreeBSD you intend to -by making it explicit. - diff --git a/mini-primer.md b/mini-primer.md deleted file mode 100644 index 015db67..0000000 --- a/mini-primer.md +++ /dev/null @@ -1,455 +0,0 @@ -# FreeBSD mini-git Primer - -## Scope -If you want to download FreeBSD, compile it from sources and generally -keep up to date that way, this primer is for you. If you are looking -to do more with the tree, contribute back, or commit changes, then you -should look HERE(link to developers guide). It covers getting the -sources, updating the sources, how to bisect and touches briefly on -how to cope with a few local changes. It covers the basics, and tries -to give good pointers to more in-depth treatment for when the readers -finds the basics insufficient. - -The goal of this document is to highlight those bits of git needed to -track sources. They assume a basic understanding of git. There are -many primers for git on the web, but the [Git -Book](https://git-scm.com/book/en/v2) provides one of the better -treatments. - -## Keeping Current With FreeBSD src tree -First step: cloning a tree. This downloads the entire tree. There are -two ways to download. Most people will want to do a deep clone of the -repo. However, there are times that you may wish to do a shallow -clone. - -### Branch names -The branch names in the new git repo are similar to the old names. For -the stable branches, they are stable/X where X is the major release -(like 11 or 12). The main branch in the new repo is 'main'. The main -branch in the old GitHub mirror is 'master'. Both reflect the -defaults of git at the time they were created. The main/master branch -is the default branch if you omit the '-b branch' or '--branch branch' -options below. - -### Repositories -Please see [URLs.md] for the latest information on where to get FreeBSD soruces. -$URL below can be obstained from that page. - -Note: The project doesn't use submodules as they are a poor fit for our workflows and development model. -How we track changes in third-party applications is discussed elsewhere and generally of little concern to the casual user. - -### Deep Clone -A deep clone pulls in the entire tree, as well as all the history and -branches. It's the easiest to do. It also allows you to use git's -worktree feature to have all your active branches checked out into -separate directories but with only one copy of the repository. -``` -% git clone -o freebsd $URL -b branch [dir] -``` -is how you make a deep clone. 'branch' should be one of the branches -listed in the previous section. It is optional if it is the -main/master branch. dir is an optional directory to place it in (the -default will be the name of the repo you are cloning (freebsd or src)). - -You'll want a deep clone if you are interested in the history, plan on -making local changes, or plan on working on more than one branch. It's -the easiest to keep up to date as well. If you are interested in the -history, but are working with only one branch and are short on space, -you can also use --single-branch to only download the one branch -(though some merge commits will not reference the merged-from branch -which may be important for some users who are interested in detailed -versions of history). - -### Shallow Clone - -A shallow clone copies just the most current code, but none or little of the history. -This can be useful when you need to build a specific revision of FreeBSD, or when you are just starting out and plan to track the tree more fully. -You can also use it to limit history to only so many revisions. -However, see below for a significant limitation of this approach. - -``` -% git clone -o freebsd -b branch --depth 1 $URL [dir] -``` - -This clones the repository, but only has the most recent version in the repository. -The rest of the history is not downloaded. -Should you change your mind later, you can do 'git fetch --unshallow' to get the old history. - -WARNING: When you make a shallow clone, you will lose the commit count in your uname output. -This can make it more difficult to determine if your system needs to be updated when a security advisory is issued. - -### Building - -Once you've downloaded, building is done as described in the handbook, -eg: -``` -% cd src -% make buildworld -% make buildkernel -% make installkernel -% make installworld -``` -so that won't be covered in depth here. - -If you want to build a custom kernel, chapter 8.4 of the FreeBSD -Handbook recommends creating a file MYKERNEL under sys/${ARCH}/conf -with your changes against GENERIC. To have MYKERNEL disregarded by -git, it can be added to .git/info/exclude. - -### Updating - -To update both types of trees uses the same commands. This pulls in -all the revisions since your last update. -``` -% git pull --ff-only -``` -will update the tree. In git, a 'fast forward' merge is one that only -needs to set a new branch pointer and doesn't need to re-create the -commits. By always doing a 'fast forward' merge/pull, you'll ensure -that you have an identical copy of the FreeBSD tree. This will be -important if you want to maintain local patches. - -See below for how to manage local changes. The simplest is to use ---autostash on the 'git pull' command, but more sophisticated options -are available. - -## Selecting a Specific Version - -In git, the 'git checkout' checks out both branches and specific -versions. Git's versions are the long hashes rather than a sequential -number. - -When you checkout a specific version, just specify the hash you want -on the command line (the git log command can help you decide which -hash you might want): -``` -% git checkout 08b8197a74 -``` -and you have that checked out. You'll be greeted with a message -similar to the following: -``` -Note: checking out '08b8197a742a96964d2924391bf9fdfeb788865d'. - -You are in 'detached HEAD' state. You can look around, make experimental -changes and commit them, and you can discard any commits you make in this -state without impacting any branches by performing another checkout. - -If you want to create a new branch to retain commits you create, you may -do so (now or later) by using -b with the checkout command again. Example: - - git checkout -b - -HEAD is now at 08b8197a742a hook gpiokeys.4 to the build -``` -where the last line is generated from the hash you are checking out -and the first line of the commit message from that revision. The hash -can be abbreviated to the shortest unique length. Git itself is -inconsistent about how many digits it displays. - -## Bisecting -Sometimes, things go wrong. The last version worked, but the one you -just updated to does not. A developer may ask to bisect the problem to -track down which commit caused the regression. - -If you've read the last section, you may be thinking to yourself "How -the heck do I bisect with crazy version numbers like that?" then this -section is for you. It's also for you if you didn't think that, but -also want to bisect. - -Fortunately, one uses the 'git bisect' command. Here's a brief outline -in how to use it. For more information, I'd suggest -https://www.metaltoad.com/blog/beginners-guide-git-bisect-process-elimination -or https://git-scm.com/docs/git-bisect for more details. The man page -is good at describing what can go wrong, what to do when versions -won't build, when you want to use terms other than 'good' and 'bad', -etc, none of which will be covered here. - -`git bisect start` will start the bisection process. Next, you need to -tell a range to go through. 'git bisect good XXXXXX' will tell it the -working version and 'git bisect bad XXXXX' will tell it the bad -version. The bad version will almost always be HEAD (a special tag for -what you have checked out). The good version will be the last one you -checked out. - -A quick aside: if you want to know the last version you checked out, -you should use 'git reflog': -``` -5ef0bd68b515 (HEAD -> master, freebsd/master, freebsd/HEAD) HEAD@{0}: pull --ff-only: Fast-forward -a8163e165c5b (upstream/master) HEAD@{1}: checkout: moving from b6fb97efb682994f59b21fe4efb3fcfc0e5b9eeb to master -... -``` -shows me moving the working tree to the master branch (a816...) and -then updating from upstream (to 5ef0...). In this case, bad would be -HEAD (or 5rf0bd68) and good would be a8163e165. As you can see from -the output, HEAD@{1} also often works, but isn't foolproof if you've -done other things to your git tree after updating, but before you -discover the need to bisect. - -Back to git bisect. Set the 'good' version first, then set the bad -(though the order doesn't matter). When you set the bad version, it -will give you some statistics on the process: -``` -% git bisect start -% git bisect good a8163e165c5b -% git bisect bad HEAD -Bisecting: 1722 revisions left to test after this (roughly 11 steps) -[c427b3158fd8225f6afc09e7e6f62326f9e4de7e] Fixup r361997 by balancing parens. Duh. -``` - -You'd then build/install that version. If it's good you'd type 'git -bisect good' otherwise 'git bisect bad'. If the version doesn't compile, -type 'git bisect skip'. You'll get a similar message -to the above after each step. When you are done, report the bad version to -the developer (or fix the bug yourself and send a patch). 'git bisect -reset' will end the process and return you back to where you started -(usually tip of main). Again, the git-bisect manual (linked above) is -a good resource for when things go wrong or for unusual cases. - -## Ports Considerations -The ports tree operates the same way. The branch names are different -and the repos are in different locations. - -The legacy GitHub mirror is at https://github.com/freebsd/freebsd-ports.git . -The canonical cgit mirror is https://cgit-beta.freebsd.org/ports.git . The -production git repo will be https://git.freebsd.org/ports.git and -ssh://anongit@git.FreeBSD.org/ports.git (or anongit@git.FreeBSD.org:ports.git) -when the time comes. - -As with ports, the 'current' branches are 'master' (legacy) and 'main' (new) -respectively. The quarterly branches are named the same as in -FreeBSD's svn repo. - -## Coping with Local Changes -This section addresses tracking local changes. If you have no local -changes, you can stop reading now (it's the last section and OK to -skip). - -One item that's important for all of them: all changes are local until -pushed. Unlike svn, git uses a distributed model. For users, for most -things, there's very little difference. However, if you have local -changes, you can use the same tool to manage them as you use to pull -in changes from FreeBSD. All changes that you've not pushed are local -and can easily be modified (git rebase, discussed below does this). - - -### Keeping local changes -The simplest way to keep local changes (especially trivial ones) is to -use 'git stash'. In its simples form, you use 'git stash' to record -the changes (which pushes them onto the stash stack). Most people use -this to save changes before updating the tree as described above. They -then use 'git stash apply' to re-apply them to the tree. The stash is -a stack of changes that can be examined with 'git stash list'. The -git-stash man page (https://git-scm.com/docs/git-stash) has all the -details. - -This method is suitable when you have tiny tweaks to the tree. When -you have anything non trivial, you'll likely be better off keeping a -local branch and rebasing. Stashing is also integrated with the 'git -pull' command: just add '--autostash' to the command line. - -### Keeping a local branch -It's much easier to keep a local branch with git than subversion. In -subversion you need to merge the commit, and resolve the -conflicts. This is manageable, but can lead to a convoluted history -that's hard to upstream should that ever be necessary, or hard to -replicate if you need to do so. Git also allows one to merge, along -with the same problems. That's one way to manage the branch, but it's -the least flexible. - -In addition to merging, git supports the concept of 'rebasing' which -avoids these issues. The 'git rebase' command replays all the commits -of a branch at a newer location on the parent branch. We'll cover the -most common scenarios that arise using it. - -#### Create a branch - -Let's say you want to make a hack to FreeBSD's ls command to never, -ever do color. There are many reasons to do this, but this example will -use that as a baseline. The FreeBSD ls command changes from time to -time, and you'll need to cope with those changes. Fortunately, with -git rebase it usually is automatic. -``` -% cd src -% git checkout main -% git checkout -b no-color-ls -% cd bin/ls -% vi ls.c # hack the changes in -% git diff # check the changes -diff --git a/bin/ls/ls.c b/bin/ls/ls.c -index 7378268867ef..cfc3f4342531 100644 ---- a/bin/ls/ls.c -+++ b/bin/ls/ls.c -@@ -66,6 +66,7 @@ __FBSDID("$FreeBSD$"); - #include - #include - #include -+#undef COLORLS - #ifdef COLORLS - #include - #include -% # these look good, make the commit... -% git commit ls.c -``` - -The commit will pop you into an editor to describe what you've -done. Once you enter that, you have your own **local** branch in the -git repo. Build and install it like you normally would, following the -directions in the handbook. git differs from other version control -systems in that you have to tell it explicitly which files to -use. I've opted to do it on the commit command line, but you can also -do it with 'git add' which many of the more in depth tutorials cover. - -#### Time to update -When it's time to bring in a new version, it's almost the same as w/o -the branches. You would update like you would above, but there's one -extra command before you update, and one after. The following assumes -you are starting with an unmodified tree. It's important to start -rebasing operations with a clean tree (git usually requires this). - -``` -% git checkout main -% git pull --no-ff -% git rebase -i main no-color-ls -``` - -This will bring up an editor that lists all the commits in it. For -this example, don't change it at all. This is typically what you are -doing while updating the baseline (though you also use the git rebase -command to curate the commits you have in the branch). - -Once you're done with the above, you've move the commits to ls.c -forward from the old version of FreeBSD to the newer one. - -Sometimes there's merge conflicts. That's OK. Don't panic. You'd -handle them the same as you would any other merge conflicts. To keep -it simple, I'll just describe a common issue you might see. A pointer -to a more complete treatment can be found at the end of this section. - -Let's say the includes changes upstream in a radical shift to terminfo -as well as a name change for the option. When you updated, you might -see something like this: -``` -Auto-merging bin/ls/ls.c -CONFLICT (content): Merge conflict in bin/ls/ls.c -error: could not apply 646e0f9cda11... no color ls -Resolve all conflicts manually, mark them as resolved with -"git add/rm ", then run "git rebase --continue". -You can instead skip this commit: run "git rebase --skip". -To abort and get back to the state before "git rebase", run "git rebase --abort". -Could not apply 646e0f9cda11... no color ls -``` - -which looks scary. If you bring up an editor, you'll see it's a -typical 3-way merge conflict resolution that you may be familiar with -from other source code systems (the rest of ls.c has been omitted): -``` -<<<<<<< HEAD -#ifdef COLORLS_NEW -#include -======= -#undef COLORLS -#ifdef COLORLS -#include ->>>>>>> 646e0f9cda11... no color ls -``` -The new code is first, and your code is second. The right fix here is -to just add a #undef COLORLS_NEW before #ifdef and then delete the old -changes: -``` -#undef COLORLS_NEW -#ifdef COLORLS_NEW -#include -``` -save the file. The rebase was interrupted, so you have to complete it: -``` -% git add ls.c -% git rebase --continue -``` - -which tells git that ls.c has been fixed and to continue the rebase -operation. Since there was a conflict, you'll get kicked into the -editor to update the commit message if necessary. It the commit -message is still accurate, just exit the editor. - -If you get stuck during the rebase, don't panic. git rebase --abort -will take you back to a clean slate. It's important, though, to start -with an unmodified tree. -An aside: The above mentioned 'git reflog' comes in handy here, as it will have -a list of all the (intermediate) commits that you can view or inspect or -cherry-pick. - -For more on this topic, -https://www.freecodecamp.org/news/the-ultimate-guide-to-git-merge-and-git-rebase/ -provides a rather extensive treatment. It is a good resource for -issues that arise occasionally but are too obscure for this guide. - -### Switching to a Different FreeBSD Branch -If you wish to shift from stable/12 to the current branch. -If you have a deep clone, the following will suffice: -``` -% git checkout main -% # build and install here... -``` -If you have a local branch, though, there's one or -two caveats. First, rebase will rewrite history, so you'll likely want -to do something to save it. Second, jumping branches tends to -encounter more conflicts. If we pretend the example above was relative -to stable/12, then to move to main, I'd suggest the following: -``` -% git checkout no-color-ls -% git checkout -b no-color-ls-stable-12 # create another name for this branch -% git rebase -i stable/12 no-color-ls --onto main -``` - -What the above does is checkout no-color-ls. Then create a new name -for it (no-color-ls-stable-12) in case you need to get back to -it. Then you rebase onto the main branch. This will find all the -commits to the current no-color-ls branch (back to where it meets up -with the stable/12 branch) and then it will replay them onto the main -branch creating a new no-color-ls branch there (which is why I had you -create a place holder name). - -### Migrating from an existing git clone -If you have work based on a previous git conversion or a locally -running git-svn conversion, migrating to new repository can encounter -problems because git has no knowledge about the connection between -the two. - -If do not have a lot of local changes, the easiest way would be to -cherry-pick your changes to the new base: -``` -% git checkout main -% git cherry-pick old_branch..your_branch -``` -Or alternatively, you can do the same thing with rebase: -``` -% git rebase --onto main master your_branch -``` - -If you do have a lot of changes, you would probably want to perform a -merge instead. The idea is to create a merge point that consolidates -the history of the old_branch, and the new source of truth (main). - -We intend to publish a set of pairs of SHA1s for this, but if you are -running a local conversion, you can find out by looking up the same -commit that are found on both parents: -``` -% git show old_branch -``` -You will see a commit message, now search for that in the new branch: -``` -% git log --grep="commit message on old_branch" freebsd/main -``` -You would get a SHA1 on the new main branch, create a helper branch -(in the example we call it 'stage') from that SHA1: -``` -% git checkout -b stage SHA1_found_from_git_log -``` -Then perform a merge of the old branch: -``` -% git merge -s ours -m "Mark old branch as merged" old_branch -``` -With that, it's possible to merge your work branch or the main branch -in any order without problem. Eventually, when you are ready to commit -your work back to main, you can perform a rebase to main, or do a -squash commit by combining everything into one commit. From 72f477ac46d98b6c733684a4e50f1208228a3cfe Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Thu, 18 Mar 2021 16:04:52 -0600 Subject: [PATCH 43/44] things pushed in --- README.adoc | 29 ++-- SUMMARY.adoc | 44 +----- commit.md | 3 - faq.md | 387 --------------------------------------------------- meta.md | 34 ----- 5 files changed, 11 insertions(+), 486 deletions(-) delete mode 100644 commit.md delete mode 100644 faq.md delete mode 100644 meta.md diff --git a/README.adoc b/README.adoc index d2c9dff..5641c7b 100644 --- a/README.adoc +++ b/README.adoc @@ -10,27 +10,14 @@ :toc: preamble -This repo contains the draft docs for the FreeBSD migration to Git. +These changes have been pushed into the FreeBSD tree. See https://docs.freebsd.org/en/articles/committers-guide/#git-primer[the committer's guide]. -== How to read the docs +Here's the few reamaining that didn't fit in: -Please check out the link:SUMMARY.adoc[SUMMARY] page for a clean and complete list of the documentation. +* link:git-why.md[FreeBSD moving to Git: Why] +* link:git-how.md[How FreeBSD implements Git] +* link:projects-user.md[A brief note about Subversion Projects and User branches] +* link:pull-request.md[How to deal with Pull Requests] +* link:big-picture.md[Big Picture] +* link:design-notes.md[Design Notes] -The best place to start is with the link:mini-primer.md[FreeBSD mini-git Primer] if -you are coming to this fresh. Otherwise the best place to start is the -link:src-cvt.md[FreeBSD Src Committer Transition Guide] if you have an existing source tree -from the project you'd like to convert over to the new git repo. Once you've -read that, you can read the link:mini-primer.md[FreeBSD mini-git Primer] for a -general primer. - -More advanced topics can be found in the link:faq.md[Git FAQ]. - -Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. - -== Contributing - -Please feel free to submit pull requests for typos, clearer language, additional -topics, etc. - -Please open issues for something that you're just noting the problem, but -don't have a solution for. diff --git a/SUMMARY.adoc b/SUMMARY.adoc index ab2734c..0d9082e 100644 --- a/SUMMARY.adoc +++ b/SUMMARY.adoc @@ -9,51 +9,13 @@ :icons: font :toc: preamble -== Preface +Most of this has been merged into https://docs.freebsd.org/en/articles/committers-guide/#git-primer[the committer's guide]. -* link:git-why.md[FreeBSD moving to Git: Why] - -== Getting Started +Here's the few reamaining that didn't fit in: +* link:git-why.md[FreeBSD moving to Git: Why] * link:git-how.md[How FreeBSD implements Git] -* https://docs.freebsd.org/en/articles/committers-guide/#git-mini-primer[FreeBSD mini-git Primer] -* https://docs.freebsd.org/en/books/handbook/mirrors/#git[Important URLs of the FreeBSD Git Infrastructure] -* https://docs.freebsd.org/en/articles/committers-guide/#git-primer[FreeBSD Src Committer Transition Guide] -* https://docs.freebsd.org/en/articles/committers-guide/#git-primer[How to do vendor imports] -* https://docs.freebsd.org/en/articles/committers-guide/#git-primer[How to MFC] * link:projects-user.md[A brief note about Subversion Projects and User branches] - -== Git References - -* link:faq.md[Git FAQ] - -== Committer References - -* https://docs.freebsd.org/en/articles/committers-guide/#commit-log-message[Writing Good FreeBSD Commit Messages] * link:pull-request.md[How to deal with Pull Requests] - -== Design & Implementation - * link:big-picture.md[Big Picture] * link:design-notes.md[Design Notes] - -== How to read the docs - -The best place to start is with the link:mini-primer.md[FreeBSD mini-git Primer] if -you are coming to this fresh. Otherwise the best place to start is the -link:src-cvt.md[FreeBSD Src Committer Transition Guide] if you have an existing source tree -from the project you'd like to convert over to the new git repo. Once you've -read that, you can read the link:mini-primer.md[FreeBSD mini-git Primer] for a -general primer. - -More advanced topics can be found in the link:faq.md[Git FAQ]. - -Other files contain information about different aspects of the src or doc repository that FreeBSD developers with commit access need to know. - -== Contributing - -Please feel free to submit pull requests for typos, clearer language, additional -topics, etc. - -Please open issues for something that you're just noting the problem, but -don't have a solution for. diff --git a/commit.md b/commit.md deleted file mode 100644 index 25f4a5b..0000000 --- a/commit.md +++ /dev/null @@ -1,3 +0,0 @@ -# Writing Good FreeBSD Commit Messages - -Now available in the diff --git a/faq.md b/faq.md deleted file mode 100644 index 86d6562..0000000 --- a/faq.md +++ /dev/null @@ -1,387 +0,0 @@ -# Git FAQ - -This document provides a number of targeted answers to questions that -are likely to come up often for users, developer and integrators. - -Note, we use the common convention of having the origin for the -FreeBSD repo being 'freebsd' rather than the default 'origin' to allow -people to use that for their own development and to minimize "whoopse" -pushes to source of truth. - -## Users - -### How do I track -current and -stable with only one copy of the repo? - -**Q:** Although disk space is not a huge issue, it's more efficient to use -only one copy of the repository. With SVN mirroring, I could checkout -multiple trees from the same repo. How do I do this with Git? - -**A:** You can use Git worktrees. There's a number of ways to do this, -but the simplest way is to use a clone to track -current, and a -worktree to track stable releases. While using a 'bare repository' -has been put forward as a way to cope, it's more complicated and will not -be documented here. - -First, you need to clone the FreeBSD repository, shown here cloning into -`freebsd-current` to reduce confusion. $URL is whatever mirror works -best for you: -``` -% git clone -o freebsd --config remote.freebsd.fetch='+refs/notes/*:refs/notes/*' $URL freebsd-current -``` -then once that's cloned, you can simply create a worktree from it: -``` -% cd freebsd-current -% git worktree add ../freebsd-stable-12 stable/12 -``` -this will checkout `stable/12` into a directory named `freebsd-stable-12` -that's a peer to the `freebsd-current` directory. Once created, it's updated -very similarly to how you might expect: -``` -% cd freebsd-current -% git checkout main -% git pull --ff-only -# changes from upstream now local and current tree updated -% cd ../freebsd-stable-12 -% git merge --ff-only freebsd/stable/12 -# now your stable/12 is up to date too -``` -I recommend using `--ff-only` because it's safer and you avoid -accidentally getting into a 'merge nightmare' where you have an extra -change in your tree, forcing a complicated merge rather than a simple -one. - -Here's https://adventurist.me/posts/00296 a good writeup that goes into more detail. -## Developers - -### Ooops! I committed to `main` instead of a branch. - -**Q:** From time to time, I goof up and commit to main instead of to a -branch. What do I do? - -**A:** First, don't panic. - -Second, don't push. In fact, you can fix almost anything if you -haven't pushed. All the answers in this section assume no push -has happened. - -The following answer assumes you committed to `main` and want to -create a branch called `issue`: -``` -% git branch issue # Create the 'issue' branch -% git reset --hard freebsd/main # Reset 'main' back to the official tip -% git checkout issue # Back to where you were -``` - -### Ooops! I committed something to the wrong branch! - -**Q:** I was working on feature on the `wilma` branch, but -accidentally committed a change relevant to the `fred` branch -in 'wilma'. What do I do? - -**A:** The answer is similar to the previous one, but with -cherry picking. This assumes there's only one commit on wilma, -but will generalize to more complicated situations. It also -assumes that it's the last commit on wilma (hence using wilma -in the `git cherry-pick` command), but that too can be generalized. - -``` -# We're on branch wilma -% git checkout fred # move to fred branch -% git cherry-pick wilma # copy the misplaced commit -% git checkout wilma # go back to wilma branch -% git reset --hard HEAD^ # move what wilma refers to back 1 commit -``` -Git experts would first rewind the wilma branch by 1 commit, switch over to -fred and then use `git reflog` to see what that 1 deleted commit was and -cherry-pick it over. - -**Q:** But what if I want to commit a few changes to `main`, but -keep the rest in `wilma` for some reason? - -**A:** The same technique above also works if you are wanting to -'land' parts of the branch you are working on into `main` before the -rest of the branch is ready (say you noticed an unrelated typo, or -fixed an incidental bug). You can cherry pick those changes into main, -then push to the parent repo. Once you've done that, cleanup couldn't -be simpler: just `git rebase -i`. Git will notice you've done -this and skip the common changes automatically (even if you had to -change the commit message or tweak the commit slightly). There's no -need to switch back to wilma to adjust it: just rebase! - -**Q:** I want to split off some changes from branch `wilma` into branch `fred` - -**A:** The more general answer would be the same as the -previous. You'd checkout/create the `fred` branch, cherry pick the -changes you want from `wilma` one at a time, then rebase `wilma` to -remove those changes you cherry picked. `git rebase -i main wilma` -will toss you into an editor, and remove the `pick` lines that -correspond to the commits you copied to `fred`. If all goes well, -and there are no conflicts, you're done. If not, you'll need to -resolve the conflicts as you go. - -The other way to do this would be to checkout `wilma` and then create -the branch `fred` to point to the same point in the tree. You can then -`git rebase -i` both these branches, selecting the changes you want in -`fred` or `wilma` by retaining the pick likes, and deleting the rest -from the editor. Some people would create a tag/branch called -`pre-split` before starting in case something goes wrong in the split, -you can undo it with the following sequence: -``` -% git checkout pre-split # Go back -% git branch -D fred # delete the fred branch -% git checkout -B wilma # reset the wilma branch -% git branch -d pre-split # Pretend it didn't happen -``` -the last step is optional. If you are going to try again to -split, you'd omit it. - -**Q:** But I did things as I read along and didn't see your advice at -the end to create a branch, and now `fred` and `wilma` are all -screwed up. How do I find what `wilma` was before I started. I don't -know how many times I moved things around. - -**A:** All is not lost. You can figure out it, so long as it hasn't -been too long, or too many commits (hundreds). - -So I created a wilma branch and committed a couple of things to it, then -decided I wanted to split it into fred and wilma. Nothing weird -happened when I did that, but let's say it did. The way to look at -what you've done is with the `git reflog`: -``` -% git reflog -6ff9c25 (HEAD -> wilma) HEAD@{0}: rebase -i (finish): returning to refs/heads/wilma -6ff9c25 (HEAD -> wilma) HEAD@{1}: rebase -i (start): checkout main -869cbd3 HEAD@{2}: rebase -i (start): checkout wilma -a6a5094 (fred) HEAD@{3}: rebase -i (finish): returning to refs/heads/fred -a6a5094 (fred) HEAD@{4}: rebase -i (pick): Encourage contributions -1ccd109 (freebsd/main, main) HEAD@{5}: rebase -i (start): checkout main -869cbd3 HEAD@{6}: rebase -i (start): checkout fred -869cbd3 HEAD@{7}: checkout: moving from wilma to fred -869cbd3 HEAD@{8}: commit: Encourage contributions -... -% -``` - -Here we see the changes I've made. You can use it to figure out where -things when wrong. I'll just point out a few things here. The first -one is that HEAD@{X} is a 'commitish' thing, so you can use that as an -argument to a command. Though if that command commits anything to the -repo, the X numbers change. You can also use the hash (first column) -as well. - -Next 'Encourage contributions' was the last commit I did to `wilma` -before I decided to split things up. You can also see the same hash is -there when I created the `fred` branch to do that. I started by -rebasing `fred` and you see the 'start', each step, and the 'finish' -for that process. While we don't need it here, you can figure out -exactly what happened. Fortunately, to fix this, you can follow the -prior answer's steps, but with the hash `869cbd3` instead of -`pre-split`. While that set of a bit verbose, it's easy to remember -since you're doing one thing at a time. You can also stack: -``` -% git checkout -B wilma 869cbd3 -% git branch -D fred -``` -and you are ready to try again. The 'checkout -B' with the hash combines -checking out and creating a branch for it. The -B instead of -b forces -the movement of a pre-existing branch. Either way works, which is what's -great (and awful) about Git. One reason I tend to use `git checkout -B xxxx hash` -instead of checking out the hash, and then creating / moving the branch -is purely to avoid the slightly distressing message about detached heads: -``` -% git checkout 869cbd3 -M faq.md -Note: checking out '869cbd3'. - -You are in 'detached HEAD' state. You can look around, make experimental -changes and commit them, and you can discard any commits you make in this -state without impacting any branches by performing another checkout. - -If you want to create a new branch to retain commits you create, you may -do so (now or later) by using -b with the checkout command again. Example: - - git checkout -b - -HEAD is now at 869cbd3 Encourage contributions -% git checkout -B wilma -``` -this produces the same effect, but I have to read a lot more and severed heads -aren't an image I like to contemplate. - -### Ooops! I did a 'git pull' and it created a merge commit, what do I do? - -**Q:** I was on autopilot and did a 'git pull' for my development tree and -that created a merge commit on the mainline. How do I recover? - -**A:** This can happen when you invoke the pull with your development branch -checked out. - -Right after the pull, you will have the new merge commit checked out. Git -supports a `HEAD^#` syntax to examine the parents of a merge commit: -``` -git log --oneline HEAD^1 # Look at the first parent's commits -git log --oneline HEAD^2 # Look at the second parent's commits -``` -From those logs, you can easily identify which commit is your development -work. Then you simply reset your branch to the corresponding `HEAD^#`: -``` -git reset --hard HEAD^2 -``` - -**Q:** But I also need to fix my 'main' branch. How do I do that? - -**A:** Git keeps track of the remote repository branches in a `freebsd/` -namespace. To fix your 'main' branch, just make it point to the remote's -'main': -``` -git branch -f main freebsd/main -``` -There's nothing magical about branches in git: they are just labels on a DAG -that are automatically moved forward by making commits. So the above works -because you're just moving a label. There's no metadata about the branch -that needs to be preserved due to this. - -### Mixing and matching branches - -**Q:** So I have two branches `worker` and `async` that I'd like to combine into one branch called `feature` -while maintaining the commits in both. - -**A:** This is a job for cherry pick. - -``` -% git checkout worker -% git checkout -b feature # create a new branch -% git cherry-pick main..async # bring in the changes -``` -You now have one branch called `feature`. This branch combines commits -from both branches. You can further curate it with `git rebase`. - -**Q:** OK Wise Guy. That was too easy. I have a branch called `driver` and I'd like -to break it up into `kernel` and `userland` so I can evolve them separately and commit -each branch as it becomes ready. - -**A:** This takes a little bit of prep work, but `git rebase` will do the heavy -lifting here. - -``` -% git checkout driver # Checkout the driver -% git checkout -b kernel # Create kernel branch -% git checkout -b userland # Create userland branch -``` -Now you have two identical branches. So, it's time to separate out the commits. -We'll assume first that all the commits in `driver` go into either the `kernel` -or the `userland` branch, but not both. - -``` -% git rebase -i main kernel -``` -and just include the changes you want (with a 'p' or 'pick' line) and -just delete the commits you don't (this sounds scary, but if worse -comes to worse, you can throw this all away and start over with the -`driver` branch since you've not yet moved it). - -``` -% git rebase -i main userland -``` -and do the same thing you did with the `kernel` branch. - -**Q:** Oh great! I followed the above and forgot a commit in the `kernel` branch. -How do I recover? - -**A:** You can use the `driver` branch to find the hash of the commit is missing and -cherry pick it. -``` -% git checkout kernel -% git log driver -% git cherry-pick $HASH -``` - -**Q:** OK. I have the same situation as the above, but my commits are all mixed up. I need -parts of one commit to go to one branch and the rest to go to the other. In fact, I have -several. Your rebase method to select sounds tricky. - -**A:** In this situation, you'd be better off to curate the original branch to separate -out the commits, and then use the above method to split the branch. - -So let's assume that there's just one commit with a clean tree. You -can either use `git rebase` with an `edit` line, or you can use this -with the commit on the tip. The steps are the same either way. The -first thing we need to do is to back up one commit while leaving the -changes uncommitted in the tree: -``` -% git reset HEAD^ -``` -Note: Do not, repeat do not, add `--hard` here since that also removes the changes from your tree. - -Now, if you are lucky, the change needing to be split up falls entirely along file lines. In that -case you can just do the usual `git add` for the files in each group than do a `git commit`. Note: -when you do this, you'll lose the commit message when you do the reset, so if you need it for -some reason, you should save a copy (though `git log $HASH` can recover it). - -If you are not lucky, you'll need to split apart files. There's another tool to do that which you -can apply one file at a time. -``` -git add -i foo/bar.c -``` -will step through the diffs, prompting you, one at time, whether to include or exclude the hunk. -Once you're done, `git commit` and you'll have the remainder in your tree. You can run it -multiple times as well, and even over multiple files (though I find it easier to do one file at a time -and use the `git rebase -i` to fold the related commits together). - -## Cloning and Mirroring - -**Q:** I'd like to mirror the entire git repo, how do I do that? - -**A:** If all you want to do is mirror, then -``` -% git clone --mirror $URL -``` -will do the trick. However, there are two disadvantages to this if you -want to use it for anything other than a mirror you'll reclone. - -First, this is a 'bare repo' which has the repository database, but no -checked out worktree. This is great for mirroring, but terrible for day to -day work. There's a number of ways around this with 'git worktree': -``` -% git clone --mirror https://cgit-beta.freebsd.org/ports.git ports.git -% cd ports.git -% git worktree add ../ports main -% git worktree add ../quarterly branches/2020Q4 -% cd ../ports -``` -But if you aren't using your mirror for further local clones, then it's a poor match. - -The second disadvantage is that Git normally rewrites the refs (branch name, tags, etc) -from upstream so that your local refs can evolve independently of -upstream. This means that you'll lose changes if you are committing to -this repo on anything other than private project branches. - -**Q:** So what can I do instead? - -**A:** Well, you can stuff all of the upstream repo's refs into a private -namespace in your local repo. Git clones everything via a 'refspec' and -the default refspec is: -``` - fetch = +refs/heads/*:refs/remotes/freebsd/* -``` -which says just fetch the branch refs. - -However, the FreeBSD repo has a number of other things in it. To see -those, you can add explicit refspecs for each ref namespace, or you -can fetch everything. To setup your repo to do that: -``` -git config --add remote.freebsd.fetch '+refs/*:refs/freebsd/*' -``` -which will put everything in the upstream repo into your local repo's -'res/freebsd/' namespace. Please note, that this -also grabs all the unconverted vendor branches and the number of refs -associated with them is quite large. - -You'll need to refer to these 'refs' with their full name because they -aren't in and of Git's regular namespaces. -``` -git log refs/freebsd/vendor/zlib/1.2.10 -``` -would look at the log for the vendor branch for zlib starting at 1.2.10. - -## Integrators diff --git a/meta.md b/meta.md deleted file mode 100644 index 68bcde2..0000000 --- a/meta.md +++ /dev/null @@ -1,34 +0,0 @@ -## Include appropriate metadata in a footer - -Commit messages may have one or more of number of standard metadata tags in the final paragraph. Standard tags used in FreeBSD are: - -| Tag | Description | -| -------- | -------- | -| PR: | FreeBSD problem report (Bugzilla) number | -| Reported-by: | ID of a 3rd party who reported the issue | -| Reviewed-by: | Reviewer ID | -| Tested-by: | ID of those who have tested the change | -| Approved-by: | Mentor or code owner who approved the change | -| Obtained-from: | Source of a change in another project | -| MFC-after: | Time period before merging the change from Current to Stable | -| MFC-with: | Associated commit that this change should be merged along with | -| MFH: | Yes/No whether ports change should be merged to quarterly branch | -| Relnotes: | Yes/No whether this change should be included in release notes | -| Security: | External reference for a security issue, such as a CVE number | -| Sponsored-by: | Organization or event that sponsored work on the change | -| Differential Revision: | Full URL of code review in FreeBSD's Phabricator instance -| Signed-off-by: | ID certifies compliance with https://developercertificate.org/ | - -"ID" indicates either a FreeBSD userid, or a name and email -address. Multiple IDs may be presented as a comma-separated list, or -by repeating metadata tags on subsequent lines. - -This represents a change from the prior FreeBSD practice of using -spaces in the metadata tags (other than in `Differential Revision`). -However, this standard conforms to the wider open-source communities use. -The project isn't requiring signed-off-by for commits at this time, but -don't want to preclude it in the future. - -In addition, you can replace - with a space in the above tags as a transition aid. - -"Submitted by" may still appear, but is generally discouraged as a legacy tag. See [How to deal with Pull Requests](pull-request.md) for details on attribution. From 3ebc4f7e1aec90671232060548f49e42e7c83134 Mon Sep 17 00:00:00 2001 From: Warner Losh Date: Thu, 8 Apr 2021 15:00:56 -0600 Subject: [PATCH 44/44] git-why is obsolete. --- README.adoc | 1 - SUMMARY.adoc | 1 - git-why.md | 200 --------------------------------------------------- 3 files changed, 202 deletions(-) delete mode 100644 git-why.md diff --git a/README.adoc b/README.adoc index 5641c7b..9a5c6a9 100644 --- a/README.adoc +++ b/README.adoc @@ -15,7 +15,6 @@ These changes have been pushed into the FreeBSD tree. See https://docs.freebsd.o Here's the few reamaining that didn't fit in: * link:git-why.md[FreeBSD moving to Git: Why] -* link:git-how.md[How FreeBSD implements Git] * link:projects-user.md[A brief note about Subversion Projects and User branches] * link:pull-request.md[How to deal with Pull Requests] * link:big-picture.md[Big Picture] diff --git a/SUMMARY.adoc b/SUMMARY.adoc index 0d9082e..6f7122f 100644 --- a/SUMMARY.adoc +++ b/SUMMARY.adoc @@ -14,7 +14,6 @@ Most of this has been merged into https://docs.freebsd.org/en/articles/committer Here's the few reamaining that didn't fit in: * link:git-why.md[FreeBSD moving to Git: Why] -* link:git-how.md[How FreeBSD implements Git] * link:projects-user.md[A brief note about Subversion Projects and User branches] * link:pull-request.md[How to deal with Pull Requests] * link:big-picture.md[Big Picture] diff --git a/git-why.md b/git-why.md deleted file mode 100644 index 8a221b0..0000000 --- a/git-why.md +++ /dev/null @@ -1,200 +0,0 @@ -# FreeBSD moving to Git: Why - -With luck, I'll be writing a few blogs on FreeBSD's move to Git later -this year. Today, we'll start with "why"? - -## Why? - -There's a number of factors motivating the change. We'll explore the -reasons, from long term viability of Subversion, to wider support for -tools that will make the project better. Today I'll enumerate these -points. There are some logistical points around how the decision was -made. I'll not get into the politics about how we got here. While -interesting for insiders who like to argue and quibble, they are no -more relevant to the larger community than the color of the delivery -truck that delivered groceries to your grocer this morning (even if it -had the latest episode of a cool, scrappy cartoon cat that was -involved in a multi-year arc wooing the love of his life by buying -food at this store). - -### Apache has moved on, so has LLVM - -The Apache Foundation used to be the care taker and main user for -Subversion. They used Subversion for all their repos. While they are -still technically the caretaker of Subversion, they've moved all their -repositories to Git. This is a worrying development because the -foreseeable outcome of this will be less Subversion development. This -will mean the FreeBSD project will need to undertake supporting -Subversion if we remain on it in the long term. FreeBSD is now the -last, large, Open Source project using Subversion. LLVM has made its -transition to Git recently. There are very real concerns about the -health and viability of the Subversion ecosystem, especially when -compared to the thriving, vibrant Git ecosystem. - -### Better CI support - -Git has more support for newer CI tools than Subversion. This will -allow us, once things are fully phased in, to increase the quality of -the code going into the tree, as well as greatly reduce build -breakages and accidental regressions. While one can use these CI tools -outside of Git, integration into a Git workflow requires less -discipline on the part of developers, making it easy for them to fix -issues found by the CI system as part of the commit/merge process -before they affect others. - -### Better merging - -Git merging facilities are much better than Subversion. You can more -easily curate patches as well since Git supports a rebase -workflow. This allows cleaner patches that are logical bits for larger -submissions. Subversion can't do this. - -Git also allows integration of multiple Git repositories with subtree -rewriting via 'git subtree merge'. This allows for easier tracking of -upstream projects and will allow us to improve the vendor import -workflow. - -### Robust mirroring - -Git can be easily and robustly be mirrored. Subversion can be -mirrored, but that mirroring is far from robust. One of the snags in -the Git migration is that different SVN mirrors have different data -than the main repo or each other. Mirroring in Git is built into the -work flow. Since every repo is cloned, mirroring comes along for -free. And there's a number of third party mirroring sites available, -such as GitHub, GitLab and SourceForge. These sites offer -collaboration and CI add-ons as well. - -Git can sign tags and commits. Subversion cannot. We can increase the -integrity of the source of truth though these measures. - -### Features from 3rd party sites - -Mirroring also opens up more 3rd party plug-ins. In terms of automated -testing and continuous integration, GitLab can do some things while -GitHub can do other things. Tests can be run when branches are -pushed. Both platforms have significant collaboration tools as well, -which will support groups going off and creating new features for -FreeBSD. While one can use these things to a limited degree, with -Subversion mirrored to GitHub, the full power of these tools isn't -fully realized without a conversion to Git. - -The wide range of tools available on these sites, or in stand-alone -configurations, will allow us to have both pre-commit checks, as well -as long-term post-commit tests running on a number of different -platforms. This will allow the project to leverage existing -infrastructure where it makes financial sense to let others run the -tests, while still allowing the project to retain control of the bits -that are critical to our operations. - -### Improved user-submitted patch tracking and integration - -One area we've struggled with as a project is patch integration. We -have no clear way to submit patches that will be acted on in a timely -fashion, or at least that's the criticism. We do have ways, but they -are only partially effective at integrating patches into the -tree. Pull requests, of various flavors, offer a centralized way to -accept patches, and have tools to review them. This should lower the -friction to people submitting patches, as well as making it easier for -developers to review the patches. Other projects have reported -increased patch flow when moving to Git. This can also be coupled with -automated testing and other validation of the patch before the -developer looks at it, which addresses one of the big issues with past -systems: very low signal to noise ratio. While not a panacea, it will -make things better and more widely use the scarce developer time. - -### Collaboration - -Some have said that Git, strictly speaking, isn't a pure source code -management system. It is really a collaboration tool that also -supports versioning. This may sound like a knock on Git, but really -it's git's greatest strength. Git's distributed model allows for -easier and more frequent collaboration. Whole webs sites have been -built around this concept, and show the power of easy collaboration to -amplify efforts and build community. - -### Skill set - -All of this is before the skill set arguments are made about kids -today just knowing Git, but needing to learn Subversion. That has -increasingly been a source of friction. This argument is supported by -anecdotal evidence of people complaining about having to learn -Subversion, about professors having to take time to teach it, etc. In -addition, studies in the industry have shown a migration to Git away -from other alternatives. Git now has between 80% and 90% of the -market, depending on which data you look at. It's so widely used that -our non-use of it is a source of friction for new developers. - -### Developers are already moving - -More and more of the active developers on the project use Git for all -or part of their development efforts. This has lead to a minor amount -of friction because all these ways are not standardized, have fit and -finish issues when pushing the changes into subversion, and cause -friction. The friction is somewhat offset by the increase in -productivity they offer these developers. The thinking is that having -Git as the source of truth will unlock the potential of Git even more -to increase productivity. - -## Downsides - -First, Git has no keyword expansion support for implementing -$FreeBSD$. While one can quibble over the Git add-ins that do this -sort of thing, the information added isn't nearly as useful as -Subversions'. This will represent a loss. However, tagged keyword -expansion has been going out of style for a long time. It used to be -that source code control systems would embed the commit history in -files committed, only to discover weird things happened when they were -imported into other projects so the practice withered. We ran into a -similar issue years ago with $Id$ and all the projects switched to -$FooBSD$ instead. This was useful when source code tracking was weak -and companies randomly imported versions: it told us what versions -were there. Companies now tend to do this with Git, which has better -tracking to the source abilities. The value isn't zero, but it's much -lower than when we adopted it in the 90s. Git also doesn't support -dealing with all the merge conflicts it causes very well. Since the -tool rewards people for importing in a more proper way, more companies -appear to be using it that way, lessening the need for explicit source -marking. [Note: some don't consider this a loss at all]. - -Second, Git doesn't have a running count of commits. One can work -around this in a number of ways, but there's nothing fundamental that -can be used as a commit number as reliably as Subversion. Even so, the -workarounds suffice for most uses and many projects are using Git and -commit counts successfully today. - -Third, the BSDL Git clients are much less mature than the GPL -ones. Until recently, there was no C client that could be imported -into the tree. While one might debate whether or not that's a good -idea, there's a strong cultural heritage of having all you need in the -base system that's hard to shrug off. OpenBSD recently wrote Game of -Trees (Got) which has an agreeable license (but a completely different -command line interface, for good or ill). It has its issues, which -aren't relevant here, but is maturing nicely. Even with the current -restrictions, it is usable. There is an active port of Got to FreeBSD -due to the large number of OpenBSDisms that are baked in (some -necessary, some gratuitous). The OpenBSD people are open to having a -portable version of Got, so this is encouraging. - -Finally, Change Is HARD. It's easy to keep using the same tools, with -the same work flows with the same people as you did -yesterday. Learning a new system is difficult. Migrating one work flow -to another is tricky. You have to balance the accumulated knowledge -and tooling benefits vs. the cost it will take to move to something -new. The Git migration team views moving from Subversion to Git as the -first step in many for improving and refining FreeBSD's work flow. As -such, we've tried to create a Git workflow that will be familiar to -old users and developers, and at the same time allow for further -innovation in the future. - -## Conclusion - -Although it's not without risk and engineering trade-offs, the bulk of -the evidence strongly suggests that moving to Git will increase our -productivity, improve project participation, grow the community, -increase quality and produce a better system in the end. It will give -us a platform that we can also re-engineer other aspects of the -project. This will come at the cost of retooling our release process, -our source distribution infrastructure and developers needing to learn -new tools. The cost will be worth it, as it will pay dividends for -years to come.