Hello Itamar,

Will be nice, and quite useful, to have this...

I fully agree with you :) @carlosmn is working hard to make this happen. Once the feature is available and tested in libgit2, we'll make sure that LibGit2Sharp benefit from it quickly.

BTW, from an API standpoint, how would you like to use it? What would be the client code you'd like to write?

Looking forward to it...

I guess new Repository(path).Pull(origin) and new
Repository(path).Fetch(origin) ?

Either this or through the Remotes API. I'm not familiar with your API
enough, but I think pulling made on a repository given a remote name makes
the most sense.

On Mon, Sep 19, 2011 at 11:49 AM, nulltoken <
[email protected]>wrote:

Hello Itamar,

Will be nice, and quite useful, to have this...

I fully agree with you :) @carlosmn is working hard to make this happen.
Once the feature is available and tested in libgit2, we'll make sure that
LibGit2Sharp benefit from it quickly.

BTW, from an API standpoint, how would you like to use it? What would be
the client code you'd like to write?

Reply to this email directly or view it on GitHub:
#65 (comment)

Looking forward to it...

I'm going to keep this issue open as a reminder. It will be updated as soon as something testable pops up.

The library already supports fetching (not every efficiently, but that'll come in good time). Personally I don't like new Repository(path).Fetch(origin), because that can block (for a few minutes if you're unlucky). What the library has now (in C) is more like

Remote origin = new Remote(path).Remote("origin");
origin.Connect();
origin.Negotiate();
if (origin.NeedsPack) {
  origin.DownloadPack();
}

All of the network operations should be run in a worker thread and you probably don't want to create another repository object since you probably already have one somewhere.

Don't mind the new stuff, I was sketching code...

My problem with this syntax is it doesn't mimic the normal git operation.
The .NET wrapper is enough high level to allow that, IMHO.

Also, what about merges? in my scenario I'm not expecting conflicts, hence
I'm interested in using Pull.

On Mon, Sep 19, 2011 at 5:43 PM, Carlos Martn Nieto <
[email protected]>wrote:

The library already supports fetching (not every efficiently, but that'll
come in good time). Personally I don't like new Repository(path).Fetch(origin), because that can block (for a few minutes
if you're unlucky). What the library has now (in C) is more like

Remote origin = new Remote(path).Remote("origin");
origin.Connect();
origin.Negotiate();
if (origin.NeedsPack) {
 origin.DownloadPack();
}

All of the network operations should be run in a worker thread and you
probably don't want to create another repository object since you probably
already have one somewhere.

Reply to this email directly or view it on GitHub:
#65 (comment)

It replicates how the git protocol works. The git command-line tool obviously does it all at once. The C# layer can certainly simplify the method all it wants to, and we may eventually take (some of) it into the library.

Merges need merge-base calculation support and the whole diff machinery, which will take a bit of time (help writing and reviewing code is always appreciated). I don't think there should be an explicit pull operation, but rather fetch + merge, both of these steps being explicitly run.

I'm too against doing just pull, but in some scenarios (like mine: auto deployment) that is more than acceptable to make. It should be easy to with with an extension function, so I don't really mind.

What is lacking in the merge and diff arena? there's plenty of OSS code that should get you jumpstarted on that one, no?

I'm too swamped with work and projects as it is, but I'd like to understand what needs to be done.

Feel free to e-mail me directly to prevent this thread from bloating.

I'm too against doing just pull, but in some scenarios (like mine: auto
deployment) that is more than acceptable to make. It should be easy to with
with an extension function, so I don't really mind.

What is lacking in the merge and diff arena? there's plenty of OSS code that
should get you jumpstarted on that one, no?

I'm too swamped with work and projects as it is, but I'd like to understand
what needs to be done.

Feel free to e-mail me directly to prevent this thread from bloating.

On Mon, Sep 19, 2011 at 6:23 PM, Carlos Martn Nieto <
[email protected]>wrote:

It replicates how the git protocol works. The git command-line tool
obviously does it all at once. The C# layer can certainly simplify the
method all it wants to, and we may eventually take (some of) it into the
library.

Merges need merge-base calculation support and the whole diff machinery,
which will take a bit of time (help writing and reviewing code is always
appreciated). I don't think there should be an explicit pull operation, but
rather fetch + merge, both of these steps being explicitly run.

Reply to this email directly or view it on GitHub:
#65 (comment)

Hey, any word on what needs to be done?

Hey, any word on what needs to be done?

Hello Itamar, sorry I've let this thread go cold.

First of all, the update of the working directory from a commit (checkout) doesn't work yet. So the Clone/Fetch experience will not be as rich as one could expect :)

Even if the API might change later (for instance, returning a [Fetch|Clone]Result or accepting callback parameters), for now, I'd go with the following pattern of usage.

using (var repo = new Repository(path))
{
   repo.Fetch(remoteAlias);
}

or

string fullPathToTheRepo = Repository.Clone(repoPath, remoteUrl);

I'd throw in some local and http Clone tests from a tiny repo within a SelfCleaningDirectory, asserting commits, refs, .... (To this purpose, I've forked @henon's TestGitRepository at https://github.com/nulltoken/TestGitRepository)

Then I'd dive in fetch.c example and bang my head in despair against the keyboard until the tests pass.

If this doesn't work, a more standard approach may help :)

• Make sure fetch.c actually works (this will of course require to build libgit2 (see this and that)
• Bind the extern'd methods in NativeMethods (depends.exe usually comes handy to troubleshoot potential issues)
• Port the C code in C#
• Make the tests pass
• Refactor
• Add some more tests
• ...

Please note, however, that this kind of task might put under the light some bugs (features-to-be?) in libgit2. When/If you encounter an issue, provided you're damn sure of your C# code, you'll eventually have to go back to C and port your non-working test from C# to C in order to give the libgit2 team something to chew while troubleshooting/fixing...

Concerning the diff, we might wait for this issue to be closed or temporarily rely on diffplex (or whatever) once the clone/fetch works.

Hope this helps,

Em.

Yay, thanks. I will have a look as soon as I can.

Relevant information in libgit2 issue #501.

So I thought I'd give it a try, and the last line of this code gives me an AccessViolationException:

var path = @"F:\Projects\Fubu\Chpokk";
var result = NativeMethods.git_repository_open(out repository, PosixPathHelper.ToPosix(path));
Assert.NotNull(repository);
Assert.AreEqual(0, result);
result = NativeMethods.git_remote_new(out remote, repository, "[email protected]:uluhonolulu/Chpokk.git", null);
Assert.AreEqual(0, result);
result = NativeMethods.git_remote_connect(remote, NativeMethods.GIT_DIR_FETCH);

My native method declarations are:

    public const int GIT_DIR_FETCH = 0;

    [DllImport(libgit2)]
    public static extern int git_remote_new(
        out RemoteSafeHandle remote,
        RepositorySafeHandle repo,
        [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(Utf8Marshaler))] string url,
        [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(Utf8Marshaler))] string name);

    [DllImport(libgit2)]
    public static extern int git_remote_connect(
        RemoteSafeHandle remote, 
        int direction);

I should admit I was too lazy to build and run the C code.

@uluhonolulu AccessViolationException confirmed. I'm going to troubleshoot this.

Meanwhile, using git://github.com/libgit2/libgit2.git or http://github.com/libgit2/libgit2.git seems to be ok.

The url [email protected]:uluhonolulu/Chpokk.git doesn't get handled by libgit2

Yay! it works!

;-)

Ok, now git_remote_download throws AccessViolationException.
The code is:

            var path = @"F:\Projects\Fubu\Chpokk";
            var result = NativeMethods.git_repository_open(out repository, PosixPathHelper.ToPosix(path));
            Assert.NotNull(repository);
            Assert.AreEqual(0, result);
            result = NativeMethods.git_remote_new(out remote, repository, "git://github.com/libgit2/libgit2.git", "origin");
            Assert.AreEqual(0, result);
            result = NativeMethods.git_remote_connect(remote, 0);
            Assert.AreEqual(0, result);
            string packname;
            result = NativeMethods.git_remote_download(out packname, remote);

Of course it might be a problem with how I declare the method:

    [DllImport(libgit2)]
    public static extern int git_remote_download(
        [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof (Utf8Marshaler))] out string filename,
        RemoteSafeHandle remote);

The original is:

GIT_EXTERN(int) git_remote_download(char **filename, git_remote *remote);

Ok, now git_remote_download throws AccessViolationException.

This reminds me of an AVE that @dahlbyk encountered in #77. Can you try and not declare filename as an out string and replace it with an IntPtr?

You can take a look at the way git_commit_message() (which also returns a string) is declared and used throughout the code. Beware that this case is slightly different as the returned commit message is a char* and the filename is a char**. You might (not sure about this) have to "unwrap" the IntPtr before invoking MarshallAsString().

@carlosmn How would you release from c# the allocated char array returned by git_remote_download()?

Do I still have to use "out" for IntPtr filename?

Both options still throw the same exception.

Both options still throw the same exception.

Hmm. You dropped the MarshalAs attribute, did you? Can you paste the stacktrace?

One last option might be to go the unsafe way with this last method by declaring a sbyte* filename and passing &filename as a parameter. You'd have to move the extern declaration from NativeMethods to UnsafeNativeMethods. You can peek at @tclem's Libgit2UnsafeHelper work for some inspiration.

    [DllImport(libgit2)]
    public static extern int git_remote_download(
        out IntPtr filename,
        RemoteSafeHandle remote);

TestCase 'LibGit2Sharp.Tests.RemoteFixture.TryToConnectToRemote'
failed: System.AccessViolationException : Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at LibGit2Sharp.Core.NativeMethods.git_remote_download(IntPtr& filename, RemoteSafeHandle remote)
RemoteFixture.cs(47,0): at LibGit2Sharp.Tests.RemoteFixture.TryToConnectToRemote()

Gonna try this unsafe thing when I have more courage, err.. time.

Could you post the exact unsafe code you want me to try? I'm a complete C noob. Also, folks suggest trying various Charset and declaration types, but CDecl doesn't work (says no entry point), what you think?

Try something like this:

private unsafe string DownloadPackFileOrWhateverVeryNiceMethodNameYouCanThinkOf(RemoteSafeHandle remoteSafeHandle)
{
    sbyte* filename;

    int result = UnSafeNativeMethods.git_remote_download(&filename, remoteSafeHandle);
    Ensure.Success(result);

    return new string(filename);
}

Adding following declaration to UnSafeNativeMethods

[DllImport(libgit2)]
public static extern int git_remote_download(sbyte** filename, RemoteSafeHandle remoteSafeHandle);

It should work.

Well, it doesn't.

failed: System.AccessViolationException : Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
at LibGit2Sharp.Core.UnSafeNativeMethods.git_remote_download(SByte** filename, RemoteSafeHandle remoteSafeHandle)
Core\UnSafeNativeMethods.cs(25,0): at LibGit2Sharp.Core.UnSafeNativeMethods.DownloadPackFileOrWhateverVeryNiceMethodNameYouCanThinkOf(RemoteSafeHandle remoteSafeHandle)
RemoteFixture.cs(47,0): at LibGit2Sharp.Tests.RemoteFixture.TryToConnectToRemote()

Couldn't it be that the actual error happens inside the native function (just like it was with the git@ path)?

Couldn't it be that the actual error happens inside the native function (just like it was with the git@ path)?

You may be right. AccessViolationException doesn't smell good. I'm going to try and troubleshoot this this evening. In order for me to be able to reproduce your context, could you push a new branch in your forked repository with the current state of your code?

BTW, previous AVE has been fixed in core libgit2 (see libgit2/libgit2@d1317f1)

It's perfectly possible that filename points to NULL. That's how the function tells you that you didn't need to download anything.

Short status update? (not trying to rush you guys :)

@kaisellgren

not trying to rush you guys :)

sure? ;p

The rework of the error handling mechanism in libgit2 kept things on hold for some time. @carlosmn has been hard at work with the streaming indexer.

As things are now a bit settled, we're fixing a few pending issues polishing the fetching code to make it perfect!

@kaisellgren for cloning, I'm using the code from @nulltoken's fork. Perfect or not, works for me. Unfortunately, I cannot submit an issue there ;).

Pushing is another issue as I understand. Needs to be implemented in libgit first. Big priority for me, but I'm incompatible with native code.

@uluhonolulu @nulltoken how are/will the private keys be handled? Does the current cloning code at @nulltoken's fork work only with public repos?

NEWS

After he recently saved a cheerleader from squishy lizards and fuzzy rodents, the amazing @BenStraub has decided to finish the job and save the world. At last. libgit2/libgit2#778 Should add native clone and checkout to libgit2.

Push is also being worked on. As @schu was a bit bored, he also decided to challenge his natural awesomeness and tackle this during the summer. With a blindfold. And a hand tied in his back. In order to keep this task a bit challenging to him.

We're going to have a pretty psychedelic library ;)

So, the libgit binaries containing the native clone implementation are now in LibGit2Sharp? Which branch?

@uluhonolulu It's not available yet. @BenStraub Is still working on it. You can peek at his work by subscribing to the notifications of libgit2/libgit2#778.

Yes I'm watching it already, thought he's improving it in the main repo.

I've been keeping this on my own fork, that seems to be more idiomatic of open-source projects on GitHub.

It's mostly ready; pull from my fork, checkout the clone branch, and try out the network example: ./git2 clone http://github.com/libgit2/libgit2 ./libgit2. Leave a note on the PR if you have feedback!

I'm a managed guy, the mere idea of having to build a native library scares me. If I could get my hands on the binary, I'd volunteer to write a managed wrapper around it. But I guess it doesn't make much sense until it's merged into the main repo.

The amazing @jamill did it! Fetch() has landed in vNext

Quick update, thanks to the amazing job done by @ben and @jamill, the following features are now available:

• Fetch()
• Clone()
• Checkout()

Are there any plans for merge support in the near future?

Are there any plans for merge support in the near future?

@jbjorge Related bits and pieces are regularly being committed by @ethomson

@jbjorge I can't answer that question without knowing your definition of "near". So instead I'll just give you a broad overview of where things are.

In short, though, yes, I hope so. It's an area that I'm actively involved in. Merge is being added to libgit2 libgit2/libgit2#1007. Once it lands there, obviously, it will land in libgit2sharp not too terribly long thereafter.

I needed to take a brief pause in order to help get the Visual Studio tooling (aka #git4vs) stuff out the door. While most everybody else was actually working, I did have to fix a few bugs in, well, merge, actually. (And I was out of the office last week.)

So you'll note an increase in activity here trying to get that branch integrated. I'm breaking git_merge_trees() out into its own PR at the moment, and you're seeing libgit2sharp reaction work like @nulltoken just pointed out. Having merge out in its own branch is a huge technical debt for us (me) and a significant source of pain for us (me) so it's certainly something that we're pushing for.

Any news here?

What's the status of this?

@powercode, @kierun As stated by @ethomson above, merge is still a work in progress.

Some lower level features have already been merged in libgit2, though (cf. libgit2/libgit2#1592 , libgit2/libgit2#1389 , libgit2/libgit2#1185 , libgit2/libgit2#, libgit2/libgit2#1011 and some other minor ones).

Some of those bits have already been made available to LibGit2Sharp (see MergeFixture, CommitFixture, ConflictFixture, CheckoutFixture).

The main PR to follow is libgit2/libgit2#1007.

I'm using LibGit2Sharp.0.13.0.0 from NuGet and understand that "Pull" isn't yet implemented, however is it possible to "mimick" a pull using different (plumbing) functions that are currently available in the API? or is my only option "shelling out" to do the pull? seems a shame when Clone() and Push() work as expected.

fetch is supported (see FetchFixture), otherwise see @nulltoken's previous comment.

Merge is still being developed in libgit2 (which would be required to do "Pull"). I believe the PR to follow is still libgit2/libgit2#1007.

Merge has landed! See #608

@synhershko ~900 days later, it looks like we may close this feature request in a near future. It looks like @jamill is making Pull() happen in #643.

🎉

@nulltoken Amazing :)

Fixed by #643