Negation of subdir ignore causes other subdirs to be unignored #5020

implausible · 2019-03-14T20:08:16Z

When a directory's contents are ignored, and then a glob negation is made to a nested subdir, other subdirectories are now unignored.

The best example I have is:

/dir/*
!/dir/sub1/sub2/**/*

After the second rule is added, files in /dir/sub1 are now unignored.

When a directory's contents are ignored, and then a glob negation is made to a nested subdir, other subdirectories are now unignored

Matching on the prefix of a negated pattern was triggering false negatives on siblings of that pattern. e.g. Given the .gitignore: dir/* !dir/sub1/sub2/** The path `dir/a.text` would not be ignored.

implausible · 2019-03-15T00:13:16Z

@pks-t It seems you were in this area last time, some guidance would be appreciated! We're seeing this problem prop up in GitKraken and wanted to try and get to the bottom of it. Let me know if there's something we're not able to grok, but it seems that the block that sat there is now no longer performing its function adequately. It seems that removal of that block is closer to the gitignore spec than the current iteration is now.

The big question I have is what do you think we broke instead?

pks-t · 2019-03-15T12:15:38Z

The big question I have is what do you think we broke instead?

This is the single most important question when it comes to gitignore code. You know, on the surface the rules look quite easy. But whenever you feel like you've improved the code to more closely match git.git, then nope. Some weird corner case you weren't aware of just broke.

To be honest, by now I don't care for what I think anymore, as I started to trust our gitignore tests a lot more than my own judgement. I'm not able to grok all those intricacies and every time I take a look atthat code I need to re-learn everything from the beginning.

Anyway, let's see where this block comes from. Blame shows f25bc0b (Fix rejection of parent dir of negated ignores, 2014-08-08):

Fix rejection of parent dir of negated ignores

While scanning through a directory hierarchy, this prevents a
positive ignore match on a parent directory from blocking the scan
of a directory when a negative match rule exists for files inside
the directory.

Unfortunately, no test has been added for that. From my guess, this looks like one of the following two scenarios:

/parent/
!/parent/child/file

We do not want to unignore "/parent/child/file" in this case. Second guess:

/parent/
!/parent/file

We do not want to unignore "/parent/child" here either.

So right now libgit2 does the right thing with regards to the above two scenarios, which might be what that commit wanted to fix. After removing the block, we still do the right thing, so it can't be required for them. Either my inferred scenarios are plain wrong or we have modified enough code for it to not be required anymore.

Long story short: your changes look quite sensible to me, at least judging from the spec.

pks-t · 2019-03-15T12:16:59Z

tests/status/ignore.c

@@ -1213,3 +1213,41 @@ void test_status_ignore__unignored_subdirs(void)
 	assert_is_ignored("dir/a.test");
 	refute_is_ignored("dir/subdir/a.test");
 }
+
+void test_status_ignore__ignored_subdirfiles_with_subdir_rule(void)


I feel like you should add these two tests to "tests/attr/ignore.c" instead.

pks-t · 2019-03-15T12:17:36Z

tests/status/ignore.c

+		"!dir/sub1/sub2/**\n");
+
+	assert_is_ignored("dir/a.test");
+	assert_is_ignored("dir/sub1/a.test");


You'd definitely want to test that "dir/sub1/sub2/foobar" is ignored, too.

pks-t · 2019-03-15T12:18:16Z

tests/status/ignore.c

+    "empty_standard_repo/dir/a.test",
+		"empty_standard_repo/dir/sub1/sub2/b.test",
+		NULL
+	};


There's no need to create those testfiles in "test/attr/ignore.c". You can just rewrite the gitignore file and then immediately use assert_is_ignored in there.

emmax86 · 2019-03-15T20:32:48Z

You are definitely on point with your suggestion that there should be a test case for "dir/sub1/sub2/foobar". Unfortunately, this iteration at the gitignore spec doesn't pass that test case; it is returns unignored, when it should be ignored. However, we were not passing this case in 0.27 or 0.28 either. It would seem this change has increased coverage of the spec without any detectable regressions. I have updated this PR with the failing assertion to demonstrate this.

For comparison, I pushed up test/ignore-0.28.1, test/ignore-0.27.8, and test/ignore-0.27.0. On these branches, I added test cases for each of the assertions in test_status_ignore__ignored_subdirfiles_with_subdir_rule to demonstrate which assertions failed. Here were my results:

0.27.0:
✅ assert_is_ignored("dir/a.test");
✅ assert_is_ignored("dir/sub1/a.test");
❌ assert_is_ignored("dir/sub1/sub2");
❌ assert_is_ignored("dir/sub1/sub2/b.test");

0.27.8
✅ assert_is_ignored("dir/a.test");
❌ assert_is_ignored("dir/sub1/a.test");
❌ assert_is_ignored("dir/sub1/sub2");
❌ assert_is_ignored("dir/sub1/sub2/b.test");

0.28.1
✅ assert_is_ignored("dir/a.test");
❌ assert_is_ignored("dir/sub1/a.test");
❌ assert_is_ignored("dir/sub1/sub2");
❌ assert_is_ignored("dir/sub1/sub2/b.test");

This PR:
✅ assert_is_ignored("dir/a.test");
✅ assert_is_ignored("dir/sub1/a.test");
✅ assert_is_ignored("dir/sub1/sub2");
❌ assert_is_ignored("dir/sub1/sub2/b.test");

implausible · 2019-03-20T20:18:56Z

@pks-t So I think if we look at the recent history of gitignore in libgit2, we're actually in a better position than we were before if this PR were to be merged. One of the selling points on merging this PR without necessarily fixing the negation case as well is that it is a vastly simpler discussion for customers. When their ignore file produces changes in GitKraken that are ignored in the CLI, if the reason is "That rule you added isn't supported in Git and should be removed from your .gitignore", then it's a super quick fix as a support issue.

Right now, the gitignore system as is produces very odd and less fixable behavior. What are your thoughts?

pks-t

Sorry it took me so long to get back. Have been scavenging off in the last few weeks to make my desktop environment workable again :/

I'm happy to merge. It seems to pass everything we'd expect it to, except for the case where we erroneously unignore files in ignored deep directories. And that case didn't work before either, so we may as well handle that in a separate PR.

implausible · 2019-03-29T14:54:11Z

Excellent. Appreciate the merge!

implausible and others added 2 commits March 14, 2019 13:06

Implement failing test for gitignore of complex subdirectory negation

bc74c53

When a directory's contents are ignored, and then a glob negation is made to a nested subdir, other subdirectories are now unignored

ignore: Do not match on prefix of negated patterns

e3d7bcc

Matching on the prefix of a negated pattern was triggering false negatives on siblings of that pattern. e.g. Given the .gitignore: dir/* !dir/sub1/sub2/** The path `dir/a.text` would not be ignored.

implausible changed the title ~~Implement failing test for gitignore of complex subdirectory negation~~ Negation of subdir ignore causes other subdirs to be unignored Mar 15, 2019

pks-t reviewed Mar 15, 2019

View reviewed changes

ignore: add additional test cases

b50e448

ignore: move tests from status to attr ignore suite

d87441f

pks-t approved these changes Mar 29, 2019

View reviewed changes

pks-t merged commit 9aa049d into libgit2:master Mar 29, 2019

implausible deleted the fix/gitignore-negation branch March 29, 2019 14:53

This was referenced Apr 3, 2019

.gitignore syntax with cascaded rules fail #5037

Closed

.gitignore cascaded negations not working correctly atom/tree-view#196

Open

.gitignore cascaded negations atom/git-utils#93

Open

pks-t added the backport-v0.28.2 label May 2, 2019

snyk-bot mentioned this pull request Feb 23, 2020

[Snyk] Upgrade nodegit from 0.4.1 to 0.26.4 saurabharch/Breezeblocks#1

Open

snyk-bot mentioned this pull request Apr 22, 2020

[Snyk] Upgrade nodegit from 0.24.3 to 0.26.5 aminatakonate000/Graviton-App#4

Open

snyk-bot mentioned this pull request May 5, 2020

[Snyk] Upgrade nodegit from 0.24.3 to 0.26.5 Barnstorm-Online/ngp-openapi-generator#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Negation of subdir ignore causes other subdirs to be unignored #5020

Negation of subdir ignore causes other subdirs to be unignored #5020

Uh oh!

implausible commented Mar 14, 2019

Uh oh!

implausible commented Mar 15, 2019 •

edited

Loading

Uh oh!

pks-t commented Mar 15, 2019

Uh oh!

pks-t Mar 15, 2019

Uh oh!

pks-t Mar 15, 2019

Uh oh!

pks-t Mar 15, 2019

Uh oh!

emmax86 commented Mar 15, 2019 •

edited

Loading

Uh oh!

implausible commented Mar 20, 2019

Uh oh!

pks-t left a comment

Uh oh!

implausible commented Mar 29, 2019

Uh oh!

Uh oh!

Negation of subdir ignore causes other subdirs to be unignored #5020

Negation of subdir ignore causes other subdirs to be unignored #5020

Uh oh!

Conversation

implausible commented Mar 14, 2019

Uh oh!

implausible commented Mar 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pks-t commented Mar 15, 2019

Uh oh!

pks-t Mar 15, 2019

Choose a reason for hiding this comment

Uh oh!

pks-t Mar 15, 2019

Choose a reason for hiding this comment

Uh oh!

pks-t Mar 15, 2019

Choose a reason for hiding this comment

Uh oh!

emmax86 commented Mar 15, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

implausible commented Mar 20, 2019

Uh oh!

pks-t left a comment

Choose a reason for hiding this comment

Uh oh!

implausible commented Mar 29, 2019

Uh oh!

Uh oh!

implausible commented Mar 15, 2019 •

edited

Loading

emmax86 commented Mar 15, 2019 •

edited

Loading