fix(vcs): git checkout after clone must contains `-- .` #2151

kamontat · 2025-05-16T09:29:56Z

Without -- . on the end cause git to not checkout latest changes from .git directory.

[fixes #1887]

… telling Git where to write out.

sisp

Thanks for fixing this bug, @kamontat ! 🙏 Would you mind adding a test case?

kamontat · 2025-06-11T08:35:47Z

@sisp I actually cannot reproduce on linux machine. The problem seem to occurred only on MacOS.

Here I tried on gitpod (follow CONTRIBUTING.md). It checkout correctly.

However, I do same thing on MacOS. It doesn't work.

Because of this I don't think test will help anything as it always pass when test on linux machine. I added anyway tho.

sisp

No problem if the error only occurs on macOS, as our CI workflows run on macOS, Linux, and Windows. But it would be good to know whether the new test fails in our macOS CI workflow without the fix. Could you temporarily push only the test case without the fix to verify this? Afterwards, if the test fails, re-add the fix to check that it makes the test pass.

sisp · 2025-06-20T07:38:42Z

tests/test_vcs.py

+@pytest.mark.impure
+def test_local_dirty_clone() -> None:
+    tmp = clone("https://github.com/copier-org/autopretty.git")
+    assert tmp
+    assert Path(tmp, "copier.yml").exists()
+
+    with local.cwd(tmp):
+        Path("unknown.txt").write_text("hello world")
+        Path("README.md").write_text("override string!!")
+
+    with pytest.warns(DirtyLocalWarning):
+        local_tmp = clone(tmp)
+
+    assert local_tmp
+    assert Path(local_tmp, "unknown.txt").exists()
+    assert Path(local_tmp, "README.md").exists()
+    assert Path(local_tmp, "copier.yaml").exists()
+    shutil.rmtree(local_tmp, ignore_errors=True)
+
+


Instead of cloning a remote repo, which makes the test impure, could you build a local repo instead? For example something like this (untested):

def test_local_dirty_clone(tmp_path_factory: pytest.TempPathFactory) -> None: src = map(tmp_path_factory.mktemp, ("src",)) build_file_tree({src / "foo.txt": "bar"}) git_save(src) build_file_tree({src / "foo.txt": "baz", src / "hello.txt": "world"}) with pytest.warns(DirtyLocalWarning): tmp = clone(src) assert tmp assert Path(tmp, "foo.txt").exists() assert Path(tmp, "foo.txt").read_text() == "baz" assert Path(tmp, "hello.txt").exists() assert Path(tmp, "hello.txt").read_text() == "world"

@sisp I revert the fixes. Please trigger CI.

kamontat · 2025-06-21T15:57:11Z

btw: those 2 commits were create from gitpod website so it not signed.

codecov · 2025-06-21T20:49:04Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.73%. Comparing base (3770a4d) to head (6d5b1e8).
⚠️ Report is 154 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2151      +/-   ##
==========================================
- Coverage   97.91%   97.73%   -0.19%     
==========================================
  Files          55       55              
  Lines        5949     6088     +139     
==========================================
+ Hits         5825     5950     +125     
- Misses        124      138      +14

Flag	Coverage Δ
unittests	`97.73% <100.00%> (-0.19%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sisp · 2025-06-22T05:38:16Z

Odd, the test case is passing on CI macOS although the fix is commented out. So, the test case doesn't seem to reproduce the problem correctly. Does the test fail on your local macOS?

kamontat · 2025-06-22T10:18:27Z

@sisp I cannot find why, but when I run clone() from test it always passed. It's not the case when I execute copier copy command.

Here is example I ran on my local machine:

Please ignore the error, I need to force to error so I can view print() result.

The output directory from tests can be checkout correctly.

However when I run copy command on dirty repository, It didn't copy template correctly.

Which after I remove dirtiness, the template is back to correct again.

Note: The red box is the print() output from _vcs#clone() function I added to verify if parameter is the same.
Note 2: I never work with python before, so now I just stuck. It seem to be some misconfigure between test environment and real environment.
Note 3: I create bash script to simulate the checkout command here

#!/usr/bin/env bash

## script.sh /path/to/template
## Expected ls -la result to contains more than .git directory

set -e

INPUT="$1"
OUTPUT="$(mktemp -d)"

rm -rf $OUTPUT

set -x

git="/usr/bin/git"

$git clone --no-checkout "$INPUT" "$OUTPUT"
cd "$OUTPUT"

$git --git-dir=.git --work-tree=$INPUT add -A
$git --git-dir=.git --work-tree=$INPUT commit -m "Copier automated commit for draft changes" --no-verify --no-gpg-sign

$git checkout -f HEAD

ls -la

Not sure if matter or not, but I have 2 git installed on my machine which I tested both of them have same issue.

kamontat · 2025-07-21T10:08:24Z

@sisp Do you need any help from me? I would love get this merge because I have to manually update copier code every time I install it.

sisp · 2025-07-21T17:43:46Z

@kamontat Sorry for the delay. I was hoping we'd find a way to test this on the macOS runner. I find it a little suspicious why running the Copier CLI is producing this error for you while the test isn't. 🤔 In the test suite, we're configuring an isolated Git config to avoid side effects from the global/system Git config. I wonder whether this might explain the difference on your machine. Not sure whether the test suite will run properly, but could you try commenting out the default_gitconfig fixture and running only your added test case?

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

kamontat · 2025-07-23T04:32:56Z

@sisp You right, now I able to reproduce failed test when core.fsmonitor is set to true. If you re-run the tests, it should fail now.

sisp · 2025-07-23T07:14:53Z

Tests on macOS and Windows are failing now – great! 🎉 Thanks for figuring this out! 🙇

You can enable the fix now, and when the tests pass, we can merge it.

kamontat · 2025-07-23T09:18:33Z

@sisp Applied hopefully the ci will pass. Thank you for your help.

sisp · 2025-07-23T09:58:58Z

Now, several other tests are failing. 🤔

kamontat · 2025-07-23T11:41:38Z

Now, several other tests are failing. 🤔

Any suggestion?

sisp · 2025-07-23T13:05:05Z

How about disabling core.fsmonitor in the local config of the cloned repository? IIUC, this feature isn't important for our use of Git.

kamontat · 2025-07-23T15:57:17Z

@sisp I need core.fsmonitor on some of my project as it significantly improve on performance.
Is it ok, if I implement try-then-fallback if repo is not clone correctly?

So it will be...

First then without --, then check
If check failed, try again with --; then check
If check failed again, then return error

sisp · 2025-07-23T16:52:55Z

Not sure if we're talking about the same approach – I might have expressed my suggestion badly. This is what I meant:

We could disable core.fsmonitor in Copier's internal template repository clone after

copier/copier/_vcs.py

Line 190 in 1e16b65

_clone()

like this:

 _clone()
+git("-C", location, "config", "--local", "core.fsmonitor", "false")

Thinking about it some more, perhaps it might be even better to disable core.fsmonitor only for the git checkout call like this:

-git("checkout", "-f", ref or "HEAD")
+git("-c", "core.fsmonitor=false", "checkout", "-f", ref or "HEAD")

WDYT?

kamontat · 2025-07-24T06:29:04Z

@sisp Oh, I see. I updated code to overrides core.fsmonitor to false on checkout command. Please rerun the test, thank you.

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

sisp · 2025-07-24T07:22:34Z

Looks like this is working. 👍 Could you add an inline comment above that line with a link to this issue, so we'll remember why the core.fsmonitor=false setting is used there? Then, I think we can merge this PR. 🎉

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

kamontat · 2025-07-24T08:41:51Z

Looks like this is working. 👍 Could you add an inline comment above that line with a link to this issue, so we'll remember why the core.fsmonitor=false setting is used there? Then, I think we can merge this PR. 🎉

@sisp Added

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

sisp

Thanks for discovering and fixing this bug, @kamontat! 🙏 I'm glad we could pin down the root cause, find a proper fix, and have it properly tested to avoid regression in the future. 🎉

fix(vcs): git checkout after clone must contains -- . to explicitly…

2c6b67a

… telling Git where to write out.

sisp requested changes May 16, 2025

View reviewed changes

chore: add more tests to dirty repository when clone

35f6e38

kamontat requested a review from sisp June 18, 2025 09:09

sisp requested changes Jun 20, 2025

View reviewed changes

kamontat added 2 commits June 21, 2025 15:54

test(vcs): update impure tests

aefbaf1

chore: revert fix to make sure it fails on CI

871f79d

chore: simulate test failed

37dbec2

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

style: autoformat with pre-commit

af61ae9

chore: apply fixed

584a9de

perf(vcs): try overrides core.fsmonitor to false when checkout

29ea2a1

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

docs: add docs on test and when checkout

6c9566d

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

docs: add github issue

6d5b1e8

Signed-off-by: Kamontat Chantrachirathumrong <[email protected]>

sisp approved these changes Jul 24, 2025

View reviewed changes

sisp merged commit ec19e44 into copier-org:master Jul 24, 2025
20 of 21 checks passed

Uh oh!

fix(vcs): git checkout after clone must contains -- . #2151

fix(vcs): git checkout after clone must contains -- . #2151

Conversation

kamontat commented May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sisp left a comment

Choose a reason for hiding this comment

Uh oh!

kamontat commented Jun 11, 2025

Uh oh!

sisp left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sisp Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

kamontat Jun 21, 2025

Choose a reason for hiding this comment

Uh oh!

kamontat commented Jun 21, 2025

Uh oh!

codecov bot commented Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sisp commented Jun 22, 2025

Uh oh!

kamontat commented Jun 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kamontat commented Jul 21, 2025

Uh oh!

sisp commented Jul 21, 2025

Uh oh!

kamontat commented Jul 23, 2025

Uh oh!

sisp commented Jul 23, 2025

Uh oh!

kamontat commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sisp commented Jul 23, 2025

Uh oh!

kamontat commented Jul 23, 2025

Uh oh!

sisp commented Jul 23, 2025

Uh oh!

kamontat commented Jul 23, 2025

Uh oh!

sisp commented Jul 23, 2025

Uh oh!

kamontat commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sisp commented Jul 24, 2025

Uh oh!

kamontat commented Jul 24, 2025

Uh oh!

sisp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fix(vcs): git checkout after clone must contains `-- .` #2151

fix(vcs): git checkout after clone must contains `-- .` #2151

kamontat commented May 16, 2025 •

edited

Loading

sisp left a comment •

edited

Loading

codecov bot commented Jun 21, 2025 •

edited

Loading

kamontat commented Jun 22, 2025 •

edited

Loading

kamontat commented Jul 23, 2025 •

edited

Loading

kamontat commented Jul 24, 2025 •

edited

Loading