Upgrade to latest libgit2 #1124

nulltoken · 2015-06-27T20:54:31Z

No description provided.

nulltoken · 2015-06-27T20:55:22Z

This has to be locally run against the NuGet package that can be downloaded here

nulltoken · 2015-06-27T20:58:48Z

The fixup commit fixes some borked tests. @Therzok can you please review the changes?

nulltoken · 2015-06-27T21:06:42Z

Remaining failing tests are submodule related

Test 'LibGit2Sharp.Tests.SubmoduleFixture.CanStageChangeInSubmoduleViaIndexStageWithOtherPaths(submodulePath: "sm_changed_head", appendPathSeparator: True)' failed:
    Assert.NotNull() Failure
    SubmoduleFixture.cs(169,0): at LibGit2Sharp.Tests.SubmoduleFixture.CanStageChangeInSubmoduleViaIndexStageWithOtherPaths(String submodulePath, Boolean appendPathSeparator)

Test 'LibGit2Sharp.Tests.SubmoduleFixture.CanStageChangeInSubmoduleViaIndexStage(submodulePath: "sm_changed_head", appendPathSeparator: True)' failed:
    Assert.NotNull() Failure
    SubmoduleFixture.cs(146,0): at LibGit2Sharp.Tests.SubmoduleFixture.CanStageChangeInSubmoduleViaIndexStage(String submodulePath, Boolean appendPathSeparator)

@dahlbyk Any chance you could peek at those and determine if our previous expectations were sound or not? Would it be some regression at the libgit2 level or some over-engineered testing at our level?

/cc @carlosmn @whoisj @jamill

nulltoken · 2015-06-27T21:07:21Z

LibGit2Sharp/StashApplyStatus.cs

@@ -19,5 +19,10 @@ public enum StashApplyStatus
        /// The stash index given was not found.
        /// </summary>
        NotFound,
+
+        /// <summary>
+        /// The stash index given was not found.


nulltoken · 2015-06-28T19:56:21Z

@dahlbyk Any chance you could peek at those and determine if our previous expectations were sound or not? Would it be some regression at the libgit2 level or some over-engineered testing at our level?

It looks like @carlosmn thinks it might be a regression as git.git seems to not choke on trailing path separators.

dahlbyk · 2015-06-29T19:13:03Z

LibGit2Sharp/LibGit2Sharp.csproj

@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
-  <Import Project="..\packages\LibGit2Sharp.NativeBinaries.1.0.55\build\LibGit2Sharp.NativeBinaries.props" Condition="Exists('..\packages\LibGit2Sharp.NativeBinaries.1.0.55\build\LibGit2Sharp.NativeBinaries.props')" />
+  <Import Project="..\packages\LibGit2Sharp.NativeBinaries.1.0.60-pre20150627175731\build\LibGit2Sharp.NativeBinaries.props" Condition="Exists('..\packages\LibGit2Sharp.NativeBinaries.1.0.60-pre20150627175731\build\LibGit2Sharp.NativeBinaries.props')" />


Version in folders = 😿

Could look into Paket to make working with Nuget easier?

I'm not to well versed in Paket. I'm really not against giving a try, but I'd rather assess it post 1.0.

@dahlbyk Those versions are managed by NuGet as part of the package update process. I'm not sure I see how you'd be able to avoid having those version numbers there since that's how NuGet works.

http://fsprojects.github.io/Paket/faq.html#NuGet-puts-the-package-version-into-the-path-Is-Paket-doing-the-same

I'm not sure that being able to remove a version number from the path is a benefit that outweighs the cost of switching to entirely different package manager. That's one more thing someone would need to understand just to be able to contribute to the project.

dahlbyk · 2015-06-29T19:18:12Z

@dahlbyk Any chance you could peek at those and determine if our previous expectations were sound or not? Would it be some regression at the libgit2 level or some over-engineered testing at our level?

It looks like @carlosmn thinks it might be a regression as git.git seems to not choke on trailing path separators.

Agreed. Easy enough to drop the trailing / in LG2S, but it's probably more correct to fix in LG2.

carlosmn · 2015-06-29T19:20:40Z

There's a PR for that! libgit2/libgit2#3269

nulltoken · 2015-06-30T22:22:51Z

Hmmm Something weird happened:

From https://ci.appveyor.com/project/libgit2/libgit2sharp/build/1091/job/jaovmnrhy66x3586

LibGit2Sharp.Tests.FilterFixture.DoubleRegistrationFailsButDoubleDeregistrationDoesNot [FAIL]
   Assert.Equal() Failure
   Expected: 0
   Actual:   1
   Stack Trace:
      c:\projects\libgit2sharp\LibGit2Sharp.Tests\FilterFixture.cs(240,0): at LibGit2Sharp.Tests.FilterFixture.DoubleRegistrationFailsButDoubleDeregistrationDoesNot()

From https://ci.appveyor.com/project/libgit2/libgit2sharp/build/1092/job/b49m75hp4dc73w2y

LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles [FAIL]
   Assert.Equal() Failure
   Expected: 134217728
   Actual:   268435456
   Stack Trace:
      c:\projects\libgit2sharp\LibGit2Sharp.Tests\FilterFixture.cs(229,0): at LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles()

.LibGit2Sharp.Tests.FilterFixture.InitCallbackMadeWhenUsingTheFilter [FAIL]
   LibGit2Sharp.EntryExistsException : A filter with the name 'the-filter' is already registered
   Stack Trace:
      c:\projects\libgit2sharp\LibGit2Sharp\Core\Proxy.cs(818,0): at LibGit2Sharp.Core.Proxy.git_filter_register(String name, IntPtr filterPtr, Int32 priority)
      c:\projects\libgit2sharp\LibGit2Sharp\FilterRegistration.cs(32,0): at LibGit2Sharp.FilterRegistration..ctor(Filter filter, Int32 priority)
      c:\projects\libgit2sharp\LibGit2Sharp\GlobalSettings.cs(233,0): at LibGit2Sharp.GlobalSettings.RegisterFilter(Filter filter, Int32 priority)
      c:\projects\libgit2sharp\LibGit2Sharp.Tests\FilterFixture.cs(86,0): at LibGit2Sharp.Tests.FilterFixture.InitCallbackMadeWhenUsingTheFilter()

This is running against latest libgit2 tip once #1113 has been merged

/cc @carlosmn @whoisj @ethomson

whoisj · 2015-06-30T23:06:49Z

This failure...

LibGit2Sharp.EntryExistsException : A filter with the name 'the-filter' is already registered

is expected because the prior test failed and did not unregister its filter.

The CanFilterLargeFiles test fails reliably for me on the ntk/upgrade branch, but I'm not 100% certain why.

whoisj · 2015-06-30T23:20:38Z

Despite messing with the test code to try various things, I keep getting the same error:

Test Name: LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles
Test FullName: LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles
Test Source: C:\Src\LibGit2Sharp\LibGit2Sharp.Tests\FilterFixture.cs : line 179
Test Outcome: Failed
Test Duration: 0:00:03.128

Result StackTrace: at LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles() in C:\Src\LibGit2Sharp\LibGit2Sharp.Tests\FilterFixture.cs:line 220
Result Message:
Assert.Equal() Failure
Expected: 134217728
Actual: 267796480

The numbers are the same every time, I'm not sure what to make of them either as these two numbers do not appear to be any kind of magic numbers.

These also happen to be very similar to the numbers AppVeyor is reporting.

LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles [FAIL]
Assert.Equal() Failure
Expected: 134217728
Actual: 268435456
Stack Trace:
c:\projects\libgit2sharp\LibGit2Sharp.Tests\FilterFixture.cs(229,0): at LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles()

Therzok · 2015-06-30T23:29:53Z

Actual is expected << 1, just sayin'.

nulltoken · 2015-07-01T05:40:12Z

Actual is expected << 1, just sayin'.

Oh! Well spotted, @Therzok. I didn't see before. I guess that's a new kind of "off by one" error 😜

nulltoken · 2015-07-01T05:56:31Z

Just pushed up the following patch

diff --git a/LibGit2Sharp.Tests/FilterFixture.cs b/LibGit2Sharp.Tests/FilterFixture.cs
index 5b89180..9f8d18f 100644
--- a/LibGit2Sharp.Tests/FilterFixture.cs
+++ b/LibGit2Sharp.Tests/FilterFixture.cs
@@ -177,7 +177,7 @@ public void WhenCheckingOutAFileFileSmudgeWritesCorrectFileToWorkingDirectory()
         [Fact]
         public void CanFilterLargeFiles()
         {
-            const int ContentLength = 128 * 1024 * 1024;
+            const int ContentLength = 128 * 1024 * 1024 + 17;
             const char ContentValue = 'x';

             char[] content = (new string(ContentValue, 1024)).ToCharArray();

Here are the new numbers

LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles [FAIL]
   Assert.Equal() Failure
   Expected: 134217745
   Actual:   268435456
   Stack Trace:
      c:\projects\libgit2sharp\LibGit2Sharp.Tests\FilterFixture.cs(229,0): at LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles()

No longer a x2 issue. Padding? Buffering related?

268435456 = 256 * 1024 * 1024

nulltoken · 2015-07-01T06:02:09Z

diff --git a/LibGit2Sharp.Tests/FilterFixture.cs b/LibGit2Sharp.Tests/FilterFixture.cs
index 9f8d18f..6092970 100644
--- a/LibGit2Sharp.Tests/FilterFixture.cs
+++ b/LibGit2Sharp.Tests/FilterFixture.cs
@@ -177,7 +177,7 @@ public void WhenCheckingOutAFileFileSmudgeWritesCorrectFileToWorkingDirectory()
         [Fact]
         public void CanFilterLargeFiles()
         {
-            const int ContentLength = 128 * 1024 * 1024 + 17;
+            const int ContentLength = 128 * 1024 * 1024 - 13;
             const char ContentValue = 'x';

             char[] content = (new string(ContentValue, 1024)).ToCharArray();

produces

LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles [FAIL]
   Assert.Equal() Failure
   Expected: 134217715
   Actual:   268433408
   Stack Trace:
      c:\projects\libgit2sharp\LibGit2Sharp.Tests\FilterFixture.cs(229,0): at LibGit2Sharp.Tests.FilterFixture.CanFilterLargeFiles()

Interestingly, 268435456 - 268433408 = 2048.

A buffer related issue somewhere?

whoisj · 2015-07-01T17:15:22Z

Well 134217728 was never (>> 1) of 267796480, so I think that it was by change we saw a fluke (nicely rounded one at that).

I'm going to compare the number of callback invocations we receive in vNext vs ntk/upgrade.

whoisj · 2015-07-01T17:30:30Z

Looks like the git_writestream.close fnp is being called more than once, for both clean and smudge.

whoisj · 2015-07-01T18:03:33Z

OK so more digging I'm seeing callbacks in this order: init, create, write (various count), close, free. The interesting part is that I'd only expect to see these calls twice (once for clean and again

When I'd only expect to see a pair of call sets: once for smudge and again for clean.

Current we see clean set, clean set, smudge set, smudge set. Thoughts?

nulltoken · 2015-07-01T18:13:38Z

/cc @shiftkey @ammeep

nulltoken · 2015-07-02T19:20:34Z

LibGit2Sharp.Tests/TestHelpers/FileExportFilter.cs

+                string filename = Path.GetFileName(path);
+                string cachePath = Path.Combine(root, ".git", filename);
+
+                if (File.Exists(cachePath))


@ethomson Is this implementation specific to this test or should something similar should always happen because of racy-git?

This is unique to this test filter. When going to the ODB, it writes a temp file with the data and returns a unique ID. Then it tests that when going to the workdir, it can replace the data with what was written in the temp file. (This test is basically a poor man's git-media or git-lfs.)

However if your filter was invoked twice (by git re-examining racily clean files, for example) then we should clean up that temp file before trying to append to it. Otherwise you would end up with it duplicated.

ethomson · 2015-07-04T15:30:44Z

@nulltoken Can we update this to the 0.23rc2? What's left to merge this?

nulltoken · 2015-07-04T16:26:16Z

Can we update this to the 0.23rc2?

@ethomson On it

libgit2/libgit2@a56db99...4d6362b

Upgrade to latest libgit2

ethomson · 2015-07-04T17:25:54Z

❤️

nulltoken reviewed Jun 27, 2015
View reviewed changes

dahlbyk reviewed Jun 29, 2015
View reviewed changes

nulltoken force-pushed the ntk/upgrade branch 2 times, most recently from 91a28d0 to 05458fe Compare June 30, 2015 22:10

nulltoken reviewed Jul 2, 2015
View reviewed changes

nulltoken force-pushed the ntk/upgrade branch 2 times, most recently from c1969a5 to c97b142 Compare July 4, 2015 16:56

nulltoken added 2 commits July 4, 2015 19:13

Make probing of test resources more reliable

4407bcb

Update libgit2 to 4d6362b

1a12ea6

libgit2/libgit2@a56db99...4d6362b

nulltoken force-pushed the ntk/upgrade branch from e208a9f to 1a12ea6 Compare July 4, 2015 17:13

nulltoken added a commit that referenced this pull request Jul 4, 2015

Merge pull request #1124 from libgit2/ntk/upgrade

37bdb63

Upgrade to latest libgit2

nulltoken merged commit 37bdb63 into vNext Jul 4, 2015

nulltoken deleted the ntk/upgrade branch July 4, 2015 17:21

nulltoken added this to the v0.22 milestone Jul 4, 2015

Upgrade to latest libgit2 #1124

Upgrade to latest libgit2 #1124

Uh oh!

Conversation

nulltoken commented Jun 27, 2015

Uh oh!

nulltoken commented Jun 27, 2015

Uh oh!

nulltoken commented Jun 27, 2015

Uh oh!

nulltoken commented Jun 27, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nulltoken commented Jun 28, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dahlbyk commented Jun 29, 2015

Uh oh!

carlosmn commented Jun 29, 2015

Uh oh!

nulltoken commented Jun 30, 2015

Uh oh!

whoisj commented Jun 30, 2015

Uh oh!

whoisj commented Jun 30, 2015

Uh oh!

Therzok commented Jun 30, 2015

Uh oh!

nulltoken commented Jul 1, 2015

Uh oh!

nulltoken commented Jul 1, 2015

Uh oh!

nulltoken commented Jul 1, 2015

Uh oh!

whoisj commented Jul 1, 2015

Uh oh!

whoisj commented Jul 1, 2015

Uh oh!

whoisj commented Jul 1, 2015

Uh oh!

nulltoken commented Jul 1, 2015

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ethomson commented Jul 4, 2015

Uh oh!

nulltoken commented Jul 4, 2015

Uh oh!

ethomson commented Jul 4, 2015

Uh oh!

Uh oh!