Codestin Search App

ethomson · 2023-08-02T21:35:24Z

Provide a smart transport that executes ssh ... - this adds a mechanism for executing processes from libgit2, and uses it to invoke ssh and deal with the output.

bergkvist · 2023-08-03T20:04:43Z

I'm getting a segfault trying to use it:

$ git clone [email protected]:bergkvist/jj && cd jj
$ cargo install --path=.
$ ~/.cargo/bin/jj init --git-repo=.
$ ~/.cargo/bin/jj git fetch
Segmentation fault

I forked/modified the following repos (Cargo.toml/submodule links only) to use this patch:

I'm getting the segfault on both macOS and Linux

ethomson · 2023-08-03T22:00:45Z

Neat, thanks for trying this out. Can you get a core or stack trace?

bergkvist · 2023-08-03T23:48:49Z

Valgrind:

valgrind ./target/debug/jj git fetch

==54547== Memcheck, a memory error detector
==54547== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==54547== Using Valgrind-3.19.0 and LibVEX; rerun with -h for copyright info
==54547== Command: ./target/debug/jj git fetch
==54547==
==54547== Invalid read of size 8
==54547==    at 0x1142164: git_transport_smart (smart.c:526)
==54547==    by 0x111AD1B: git_transport_new (transport.c:134)
==54547==    by 0x1103E77: git_remote_connect_ext (remote.c:960)
==54547==    by 0x11049CB: connect_or_reset_options (remote.c:1254)
==54547==    by 0x1104D5F: git_remote_download (remote.c:1339)
==54547==    by 0x82E8BF: git2::remote::Remote::download (remote.rs:249)
==54547==    by 0x87951B: jj_lib::git::fetch (git.rs:675)
==54547==    by 0x46AD5F: jj_cli::commands::git::cmd_git_fetch::{{closure}} (git.rs:303)
==54547==    by 0x469683: jj_cli::commands::git::with_remote_callbacks (git.rs:522)
==54547==    by 0x3FBF23: jj_cli::commands::git::cmd_git_fetch (git.rs:302)
==54547==    by 0x3F31D7: jj_cli::commands::git::cmd_git (git.rs:1027)
==54547==    by 0x276337: jj_cli::commands::run_command (mod.rs:3743)
==54547==  Address 0x10 is not stack'd, malloc'd or (recently) free'd
==54547==
==54547== Jump to the invalid address stated on the next line
==54547==    at 0x0: ???
==54547==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==54547==
==54547==
==54547== Process terminating with default action of signal 11 (SIGSEGV)
==54547==  Bad permissions for mapped region at address 0x0
==54547==    at 0x0: ???
==54547==
==54547== HEAP SUMMARY:
==54547==     in use at exit: 2,480,221 bytes in 23,448 blocks
==54547==   total heap usage: 336,841 allocs, 313,393 frees, 22,545,464 bytes allocated
==54547==
==54547== LEAK SUMMARY:
==54547==    definitely lost: 0 bytes in 0 blocks
==54547==    indirectly lost: 0 bytes in 0 blocks
==54547==      possibly lost: 182,032 bytes in 1,613 blocks
==54547==    still reachable: 2,298,189 bytes in 21,835 blocks
==54547==         suppressed: 0 bytes in 0 blocks
==54547== Rerun with --leak-check=full to see details of leaked memory
==54547==
==54547== For lists of detected and suppressed errors, rerun with: -s
==54547== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)
Segmentation fault

bergkvist · 2023-08-04T00:07:29Z

Commenting out the t->wrapped->free(t->wrapped); in smart.c:526 - I instead get:

Error: cannot create SSH transport; library was built without SSH support; class=Invalid (3)

ethomson · 2023-08-04T08:18:38Z

Oops, that's definitely a mistake. It looks like I haven't built that branch without any SSH support in a while. Thanks for catching that.

You'll need to configure libgit2 with cmake -DUSE_SSH=exec <other args> when you compile it. Sorry, I should have mentioned that upthread.

bergkvist · 2023-08-09T01:45:57Z

I had to do this to get it working: bergkvist/git2-rs@9a4d232 (git2-rs doesn't use cmake)

I would love for this to get merged into libgit2.

ethomson · 2023-08-09T20:44:04Z

I would love for this to get merged into libgit2.

It'll happen - I think that I just want to give plenty of time for feedback. Doing an exec could be new and scary for some people, even if it's only conditionally compiled in.

We may want to support SSH but with a different provider that is not libssh2. Add GIT_SSH to indicate that we have some inbuilt SSH support and GIT_SSH_LIBSSH2 to indicate that support is via libssh2. This is similar to how we support GIT_HTTPS and GIT_OPENSSL, for example.

We can now use the `git_process` class to invoke OpenSSH and use it as an SSH transport. This may be preferred over libssh2 for a variety of callers.

We can't reliably detect SIGPIPE on close because of platform differences. Track `pid` and send `SIGTERM` to a function and ensure that we can detect it.

There are no custom callbacks for OpenSSH; don't test them.

Now that we (may) exec a child process to do ssh, we don't want valgrind reporting on that. Suppress children in valgrind runs.

A transport may want to validate that it's in a sane state; when flushing on close, don't assume that we're doing an upload-pack; send the correct direction.

Instead of "early EOF", provide information on _when_ we're seeing the EOF for debugging.

Suppress SIGPIPEs during writes to our piped process. On single-threaded applications, this is as simple as ignoring the signal. But since this is process-wide, on multi-threaded applications, we need to use some cumbersome `pthread_sigmask` manipulation. Thanks to https://www.doof.me.uk/2020/09/23/sigpipe-and-how-to-ignore-it/ and http://www.microhowto.info:80/howto/ignore_sigpipe_without_affecting_other_threads_in_a_process.html

Provide a mechanism for callers to read from stderr.

Provide more user-friendly error messages in smart protocol negotiation failures.

Don't capture stderr, optimize for the CLI case.

Provide both cmdline-style handling (passing it to the shell on POSIX, or directly to CreateProcess on win32) and execv style (passing it directly to execv on POSIX, and mangling it into a single command-line on win32).

Callers can specify the ssh command to invoke using `core.sshcommand` or the `GIT_SSH` environment variable. This is useful for specifying alternate configuration, and is particularly useful for our testing environment.

This helped when troubleshooting issues running the `ci/test.sh` script locally.

Handle custom paths for OpenSSH.

This commit changes the original `ssh` feature into two new ones: `ssh-libssh2` and `ssh-openssh`. By default, the `ssh-libssh2` feature is enabled for backwards compatibility. To use OpenSSH instead, the following listing in `Cargo.toml` can be used: git2-rs = { version = "...", default-features = false, features = ["https", "ssh-openssh"] } Note that libgit2/libgit2#6617 has not actually been released in an official libgit2 version, so the prior commit pulled in the latest commit from `main`. Closes rust-lang#1028.

ethomson mentioned this pull request Aug 3, 2023

Support for SSH ProxyCommand #4872

Closed

bergkvist mentioned this pull request Aug 3, 2023

git: SSH config not respected jj-vcs/jj#63

Closed

bergkvist added a commit to bergkvist/git2-rs that referenced this pull request Aug 3, 2023

Update libgit2 to use libgit2/libgit2#6617

f40c271

ethomson added 20 commits August 30, 2023 20:45

Introduce git_strlist to manage string lists

0181b4c

Introduce git_process class that invokes processes

bc97d01

ssh: refactor libssh2-specific bits into their own file

67ab8f0

ssh: introduce GIT_SSH_EXEC for external OpenSSH

a8b052f

We can now use the `git_process` class to invoke OpenSSH and use it as an SSH transport. This may be preferred over libssh2 for a variety of callers.

ci: split ssh into exec and libssh2

38b16b0

process: test SIGTERM detection

7a8c4d8

We can't reliably detect SIGPIPE on close because of platform differences. Track `pid` and send `SIGTERM` to a function and ensure that we can detect it.

tests: use cl_fail_with correctly

cf133a8

ssh: only test callbacks for libssh2

10fbb5c

There are no custom callbacks for OpenSSH; don't test them.

ci: valgrind shouldn't report on exited children

a283c1b

Now that we (may) exec a child process to do ssh, we don't want valgrind reporting on that. Suppress children in valgrind runs.

smart: don't assume directionality

1c72c59

A transport may want to validate that it's in a sane state; when flushing on close, don't assume that we're doing an upload-pack; send the correct direction.

smart: unique error messages

0e62615

Instead of "early EOF", provide information on _when_ we're seeing the EOF for debugging.

process: reader for stderr

18474f7

Provide a mechanism for callers to read from stderr.

transport: provide clearer / unique error messages

bc124bb

Provide more user-friendly error messages in smart protocol negotiation failures.

ssh: don't capture stderr

23424f4

Don't capture stderr, optimize for the CLI case.

process: provide cmdline-handling and execv style

d7060af

Provide both cmdline-style handling (passing it to the shell on POSIX, or directly to CreateProcess on win32) and execv style (passing it directly to execv on POSIX, and mangling it into a single command-line on win32).

ssh: honor core.sshcommand

019cf6c

Callers can specify the ssh command to invoke using `core.sshcommand` or the `GIT_SSH` environment variable. This is useful for specifying alternate configuration, and is particularly useful for our testing environment.

ci: specify full path to certificate/key

3eb7ff2

This helped when troubleshooting issues running the `ci/test.sh` script locally.

ssh: exec with paths for OpenSSH

f26d92a

Handle custom paths for OpenSSH.

branchv mentioned this pull request Mar 21, 2024

libgit2 1.8.1 [email protected] 1.7.2 (new formula) Homebrew/homebrew-core#166767

Merged

razielanarki mentioned this pull request Apr 19, 2024

[libgit2] Update to 1.8.0 microsoft/vcpkg#38169

Merged

7 tasks

danielcompton mentioned this pull request Dec 2, 2024

Support SSH signing git-up/GitUp#1036

Open

This was referenced Feb 20, 2025

Question: SSH Support libgit2/libgit2sharp#1422

Closed

using SSH instead of username\password libgit2/libgit2sharp#1881

Closed

This was referenced Oct 27, 2025

chore: Bump LibGit2Sharp from 0.30.0 to 0.31.0 aweXpect/aweXpect.Chronology#74

Open

chore: Bump LibGit2Sharp from 0.30.0 to 0.31.0 aweXpect/aweXpect.Testably#107

Open

chore: Bump LibGit2Sharp from 0.30.0 to 0.31.0 aweXpect/aweXpect.Reflection#165

Open

dependabot bot mentioned this pull request Nov 5, 2025

Bump LibGit2Sharp from 0.30.0 to 0.31.0 ATrefzer/CSharpCodeAnalyst#39

Merged

dependabot bot mentioned this pull request Nov 24, 2025

chore: Bump LibGit2Sharp from 0.30.0 to 0.31.0 aweXpect/aweXpect.Mockolate#25

Merged

dependabot bot mentioned this pull request Dec 3, 2025

Bump BenchmarkDotNet and 4 others leonardochaia/dotnet-affected#131

Closed

This was referenced Dec 10, 2025

Bump BenchmarkDotNet and 4 others leonardochaia/dotnet-affected#132

Closed

Bump BenchmarkDotNet and 5 others leonardochaia/dotnet-affected#133

Closed

Bump the dotnet group with 8 updates leonardochaia/dotnet-affected#134

Closed

This was referenced Dec 24, 2025

Bump the dotnet group with 7 updates leonardochaia/dotnet-affected#135

Closed

Bump the dotnet group with 9 updates leonardochaia/dotnet-affected#136

Closed

This was referenced Dec 31, 2025

Bump BenchmarkDotNet and 6 others leonardochaia/dotnet-affected#137

Closed

Bump BenchmarkDotNet and 5 others leonardochaia/dotnet-affected#138

Closed

Bump the dotnet group with 8 updates leonardochaia/dotnet-affected#139

Closed

dependabot bot mentioned this pull request Jan 19, 2026

[dependabot]: Bump LibGit2Sharp from 0.30.0 to 0.31.0 dotnet/docker-tools#1937

Merged

This was referenced Jan 28, 2026

Bump the dotnet group with 8 updates leonardochaia/dotnet-affected#143

Closed

Bump LibGit2Sharp from 0.30.0 to 0.31.0 dotnet/source-indexer#241

Open

Bump LibGit2Sharp from 0.30.0 to 0.31.0 radical/source-indexer#6

Open

dependabot bot mentioned this pull request Feb 11, 2026

Bump the dotnet group with 8 updates leonardochaia/dotnet-affected#144

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add OpenSSH support#6617

Add OpenSSH support#6617
ethomson merged 21 commits intomainfrom
ethomson/openssh

ethomson commented Aug 2, 2023

Uh oh!

bergkvist commented Aug 3, 2023 •

edited

Loading

Uh oh!

ethomson commented Aug 3, 2023

Uh oh!

bergkvist commented Aug 3, 2023

Uh oh!

bergkvist commented Aug 4, 2023

Uh oh!

ethomson commented Aug 4, 2023

Uh oh!

bergkvist commented Aug 9, 2023

Uh oh!

ethomson commented Aug 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ethomson commented Aug 2, 2023

Uh oh!

bergkvist commented Aug 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ethomson commented Aug 3, 2023

Uh oh!

bergkvist commented Aug 3, 2023

Uh oh!

bergkvist commented Aug 4, 2023

Uh oh!

ethomson commented Aug 4, 2023

Uh oh!

bergkvist commented Aug 9, 2023

Uh oh!

ethomson commented Aug 9, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bergkvist commented Aug 3, 2023 •

edited

Loading