Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Repository.Clone with RecurseSubmodules crashes for nested submodule #1115

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
philmccarthy24 opened this issue Jun 24, 2015 · 17 comments
Closed

Comments

@philmccarthy24
Copy link

Using LibGit2Sharp 0.22.0-pre20150606092322, I cloned a repository containing a submodule, which in turn also has a submodule. I'm using CloneOptions with RecurseSubmodules=true.

The Repository.Clone call crashes with the following exception:

LibGit2Sharp.RecurseSubmodulesException was unhandled
  HResult=-2146233088
  Message=The top level repository was cloned, but there was an error cloning its submodules.
  Source=LibGit2Sharp
  InitialRepositoryPath=C:\Workspace\ASAClone\.git\
  StackTrace:
       at LibGit2Sharp.Repository.Clone(String sourceUrl, String workdirPath, CloneOptions options) in c:\projects\libgit2sharp\LibGit2Sharp\Repository.cs:line 661
       at ASABuildAgent.GitCheckoutInstruction.Execute(Stream outputLogStream) in c:\Workspace\asa\Source\ToolsAndUtils\Build\ASABuildAgent\GitCheckoutInstruction.cs:line 44
       at ASABuildAgent.BuildAgent.DoBuild() in c:\Workspace\asa\Source\ToolsAndUtils\Build\ASABuildAgent\BuildAgent.cs:line 37
       at ASABuildAgent.Program.Main(String[] args) in c:\Workspace\asa\Source\ToolsAndUtils\Build\ASABuildAgent\Program.cs:line 43
       at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
       at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
  InnerException: LibGit2Sharp.LibGit2SharpException
       HResult=-2146233088
       Message=Cannot checkout to invalid path 'boost/accumulators/accumulators.hpp'
       Source=LibGit2Sharp
       StackTrace:
            at LibGit2Sharp.Core.Ensure.HandleError(Int32 result) in c:\projects\libgit2sharp\LibGit2Sharp\Core\Ensure.cs:line 127
            at LibGit2Sharp.Core.Ensure.ZeroResult(Int32 result) in c:\projects\libgit2sharp\LibGit2Sharp\Core\Ensure.cs:line 145
            at LibGit2Sharp.Core.Proxy.git_submodule_update(SubmoduleSafeHandle submodule, Boolean init, GitSubmoduleOptions& options) in c:\projects\libgit2sharp\LibGit2Sharp\Core\Proxy.cs:line 2565
            at LibGit2Sharp.SubmoduleCollection.Update(String name, SubmoduleUpdateOptions options) in c:\projects\libgit2sharp\LibGit2Sharp\SubmoduleCollection.cs:line 114
            at LibGit2Sharp.Repository.RecursivelyCloneSubmodules(CloneOptions options, String repoPath, Int32 recursionDepth) in c:\projects\libgit2sharp\LibGit2Sharp\Repository.cs:line 721
            at LibGit2Sharp.Repository.RecursivelyCloneSubmodules(CloneOptions options, String repoPath, Int32 recursionDepth) in c:\projects\libgit2sharp\LibGit2Sharp\Repository.cs:line 735
            at LibGit2Sharp.Repository.Clone(String sourceUrl, String workdirPath, CloneOptions options) in c:\projects\libgit2sharp\LibGit2Sharp\Repository.cs:line 657
       InnerException: 
@nulltoken
Copy link
Member

🆒 report ! Thanks for this. Some questions below:

  • Can you please confirm that git.exe has no issue cloning this repo?
  • The inner exception states Cannot checkout to invalid path 'boost/accumulators/accumulators.hpp'. Could you please take a look at what this path point to? Is there some symlink involved or anything that may seem unusual?
  • By any chance, would the repository you're encountering this issue with be public (so that we could repro this issue more easily).

/cc @jamill

@philmccarthy24
Copy link
Author

Hi - thanks for the speedy response. 😄
I can clone the repo using standard git bash, ie do a git clone and then
git submodule update --init --recursive
No symlinks involved, it's just a repository (containing boost) as a submodule in a library repository that's then included as a submodule of my repo. Looks like it's the very first file that would be cloned in the nested submodule that's failing.
I'm afraid it's a private repo - sorry, I know this makes it harder to look into. I'd guess you could reproduce by creating a small repo with a couple of files, then adding a submodule, then adding that to another repository as a submodule.

The snippet of libgit2sharp code I'm using to do the cloning is:

Repository.Clone(GitRepository, WorkingDirectory,
   new CloneOptions()
      {
         RecurseSubmodules = true,
         BranchName = this.CheckoutBranch,
         CredentialsProvider = (_url, _user, _cred) => new UsernamePasswordCredentials
         {
            Username = GitUsername,
            Password = GitPassword
         },
         OnCheckoutProgress = checkoutProgress,
         OnProgress = progressHandler,
      });

Hope that helps!

@nulltoken
Copy link
Member

@philmccarthy24 Thanks for the prompt feedback as well 😉

I can clone the repo using standard git bash, ie do a git clone and then git submodule update --init --recursive

I imagine that the plain git clone --recursive works as well?

As far I can tell, @jamill made some extensive tests while teaching clone to honor --recursive. During those tests we've stumbled upon some pretty weird repo setups. However, it's fairly possible we've missed some corner cases. As such, we'd be very interested in fixing your issue.

No symlinks involved, it's just a repository (containing boost) as a submodule in a library repository that's then included as a submodule of my repo. Looks like it's the very first file that would be cloned in the nested submodule that's failing.

Any chance, you could publish a very simple public repository with such a configuration so that we could start working on this? Without it, I'm afraid it may take some time before we're able to find by chance a potential matching setup.

@philmccarthy24
Copy link
Author

Bah! I've tried to replicate the issue using a simple public repo on github with nested submodules:
https://github.com/philmccarthy24/LibGit2SharpTest.git
However that doesn't fail in the way observed above.
Sorry can't give a simple case you can replicate the issue at the moment (so will close this - maybe just be aware in future that something slightly hokey could be going on during recursive submodule checkout!)

@nulltoken
Copy link
Member

@philmccarthy24 Thanks for having given a try.

One thing though, could it be related to boost. If you submodule boost in your test repository, are you able to repro this?

Just reopened the issue in order to keep track of this.

@jamill
Copy link
Member

jamill commented Jun 24, 2015

Message=Cannot checkout to invalid path 'boost/accumulators/accumulators.hpp'

This indicates a problem with the path verification logic (that verifies that no one is trying to write into the .git directory). Maybe @ethomson would have some thoughts on what might be going on here. If we cannot figure out what is going on from the current information, we could always build an instrumented libgit2to log what is failing validation here.

@ethomson
Copy link
Member

I would first check the length of the path that file would have when
written to disk. Especially with nested submodules, you can get to
MAX_PATH pretty quickly.

On Wed, Jun 24, 2015 at 9:01 AM, Jameson Miller [email protected]
wrote:

Message=Cannot checkout to invalid path
'boost/accumulators/accumulators.hpp'

This indicates a problem with the path verification logic (that verifies
that no one is trying to write into the .git directory). Maybe @ethomson
https://github.com/ethomson would have some thoughts on what might be
going on here. If we cannot figure out what is going on from the current
information, we could always build an instrumented libgit2to log what is
failing validation here.


Reply to this email directly or view it on GitHub
#1115 (comment)
.

@philmccarthy24
Copy link
Author

Hi all
If it helps, the path of the file that was failing to be checked out was:
"C:\Workspace\asa\submodules\pplus\submodules\boost\boost\accumulators\accumulators.hpp"
Sorry as this is work I don't have more time to contribute - I've got a plan B so will have to move on to that.
Thanks for the help and good luck with the issue.
All the best, Phil

@whoisj
Copy link

whoisj commented Jun 25, 2015

86 < 259 so MAX_PATH doesn't seem to be the problem.

Wondering if there are junctions or hard-links in there anywhere.

Sorry as this is work I don't have more time to contribute

Guess we'll never know 😞

@nulltoken
Copy link
Member

@philmccarthy24 Would the pplus submodule be an open source project?

@embix
Copy link

embix commented Nov 26, 2015

to reproduce the bug try cloning this repo recursivly: https://github.com/embix/bug_libgit2sharpssubmodules/
(which works fine with git clone --recursive)

@ethomson
Copy link
Member

@embix - This sounds like libgit2/libgit2#3500

What version of LibGit2Sharp are you using? Can you try with a nightly?

@embix
Copy link

embix commented Nov 26, 2015

LibGit2Sharp 14.0.23011.0 as provided with TFS 2015 RTM vNext Build Agent for Windows LibGit2Sharp_hash.txt says dcc2ef27218939d4f13652dd12d1716bac871eea.

I guess the version does not help, as it is they use the VS version instead of the real libgit2sharp version.
I haven't tried with a nightly so far but I am willing to do so.

@ethomson
Copy link
Member

I see. The versions included in Visual Studio and TFS are quite different than released versions. Although Microsoft works closely with the LibGit2Sharp and libgit2 projects, these are different branches. And you couldn't drop a nightly into the build agent for a number of reasons.

This should be fixed in TFS 2015 Update 1. I think that this fix made it into the RC build if you want to try a beta of the next build agent: https://www.visualstudio.com/en-us/news/tfs2015-update1-vs.aspx

-ed

On Nov 26, 2015, at 9:51 AM, Michel Bretschneider [email protected] wrote:

LibGit2Sharp 14.0.23011.0 as provided with TFS 2015 RTM vNext Build Agent for Windows LibGit2Sharp_hash.txt says dcc2ef27218939d4f13652dd12d1716bac871eea.

Haven't tried with a nightly so far but willing to do so.


Reply to this email directly or view it on GitHub.

@embix
Copy link

embix commented Nov 27, 2015

sounds promising, thx

@embix
Copy link

embix commented Dec 4, 2015

Confirmed. Update 1 fixed that bug.

@ethomson
Copy link
Member

ethomson commented Dec 4, 2015

Fixed via libgit2/libgit2#3500

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants