Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add support for homedir expansion in SSL key and cert paths#5657

Merged
bk2204 merged 3 commits into
git-lfs:mainfrom
bk2204:cert-homedir-fixes
Feb 26, 2024
Merged

Add support for homedir expansion in SSL key and cert paths#5657
bk2204 merged 3 commits into
git-lfs:mainfrom
bk2204:cert-homedir-fixes

Conversation

@bk2204

@bk2204 bk2204 commented Feb 23, 2024

Copy link
Copy Markdown
Member

Git accepts paths for SSL keys and certs starting with a tilde to expand to the user's home directory. This is very useful for configuration which must be shared across systems. Let's add support for the same thing here and add a test to make sure it works.

In addition, avoid crashing on invalid data by verifying our PEM block before attempting to decode it.

Fixes #5652

Right now, our certificate handling silently ignores any problems and
just pretends like no certificate was specified.  We do log these values
to the trace output, but it's hard for people to find out what's going
on and correct the problem.

Let's add some error handling here so that people know what's going
wrong and can act on it.  In this particular case, we include specific
information in the key file that indicates that we only accept PKCS#1
files to help the user debug.
Right now, if our certificate data is not a valid PEM block, we return
nil in `block`, and then crash when checking to see if it's encrypted.
Let's fix this by checking to see if the block is nil, and if so,
returning an error.
Git accepts paths for SSL keys and certs starting with a tilde to expand
to the user's home directory.  This is very useful for configuration
which must be shared across systems.  Let's add support for the same
thing here and add a test to make sure it works.

Skip the test on Windows, because Git for Windows returns “fatal:
refusing to work with credential missing host field” when it sees the
fake `cert:///` URL without a hostname that's used to query the
credential help for our TLS certificate credentials.
@bk2204 bk2204 force-pushed the cert-homedir-fixes branch from 55b2bba to c54c9da Compare February 23, 2024 18:12
@bk2204 bk2204 marked this pull request as ready for review February 26, 2024 14:34
@bk2204 bk2204 requested a review from a team as a code owner February 26, 2024 14:34

@chrisd8088 chrisd8088 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thank you!

@bk2204 bk2204 merged commit 7064c09 into git-lfs:main Feb 26, 2024
@bk2204 bk2204 deleted the cert-homedir-fixes branch February 26, 2024 16:40
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 1, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the http.sslCert and http.sslKey Git configuration
values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
http.sslCert and http.sslKey Git configuration options to paths
beginning with a tilde, in order to validate our support for such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
http.sslCert and http.sslKey Git configuration options to their usual
values; that is, to the values set by the setup() shell function run
at the start of each test file.  This makes sense, because the goal
was to ensure that in all cases, the tests following the new "clone
ClientCert with homedir certs" test had the expected values for these
configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into
the home directory used for each test.  So we can simply remove
the two "git config" calls at the start of the "clone with flags" test,
as the values they set are always the same as the ones provided by
our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 3, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the http.sslCert and http.sslKey Git configuration
values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
http.sslCert and http.sslKey Git configuration options to paths
beginning with a tilde, in order to validate our support for such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
http.sslCert and http.sslKey Git configuration options to their usual
values; that is, to the values set by the setup() shell function run
at the start of each test file.  This makes sense, because the goal
was to ensure that in all cases, the tests following the new "clone
ClientCert with homedir certs" test had the expected values for these
configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into
the home directory used for each test.  So we can simply remove
the two "git config" calls at the start of the "clone with flags" test,
as the values they set are always the same as the ones provided by
our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 6, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the http.sslCert and http.sslKey Git configuration
values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
http.sslCert and http.sslKey Git configuration options to paths
beginning with a tilde, in order to validate our support for such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
http.sslCert and http.sslKey Git configuration options to their usual
values; that is, to the values set by the setup() shell function run
at the start of each test file.  This makes sense, because the goal
was to ensure that in all cases, the tests following the new "clone
ClientCert with homedir certs" test had the expected values for these
configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into
the home directory used for each test.  So we can simply remove
the two "git config" calls at the start of the "clone with flags" test,
as the values they set are always the same as the ones provided by
our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 10, 2024
Due to the issue described in git-lfs#5658, the "cloneSSL" and "clone ClientCert"
tests in our t/t-clone.sh test script do not run to completion but exit
early, as a consequence of an improper check of the TRAVIS variable (which
is no longer used since we migrated our test suite to GitHub Actions in
PR git-lfs#3808).

As well, since the "clone ClientCert with homedir certs" test was added
in commit c54c9da, as part of PR git-lfs#5657,
it has skipped executing on Windows due to an error that was seen when
Git attempted to retrieve the passphrase for the encrypted key of the
TLS certificate used in that test from our git-credential-lfstest
helper program.

We expect to address the first issue in subsequent commit in this PR,
at which point the tests will run fully, exposing a number of
long-standing problems.

In the case of the "cloneSSL" test, Schannel's CertGetNameString() function
will reject the TLS certificate of our lfstest-gitserver program because
we connect to the server on the local IP address 127.0.0.1, and although
the certificate lists that address as an IPAddress field among its Subject
Alternative Name (SAN) attributes, the Schannel library only looks for
hostname matches with the DNSName fields and ignores any IPAddress fields.
As a result, Git returns a "failed to match connection hostname (127.0.0.1)
against server certificate names" error, as produced by libcurl.  As of
version 8.9.0 of libcurl, the CertGetNameString() function is called
with the CERT_NAME_DNS_TYPE option, which is documented to have this
behaviour:

  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L577-L582
  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L373-L378
  https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certgetnamestringw#cert_name_dns_type

The TLS certificate presented by our lfstest-gitserver program is the
one provided by the Go language's net/http/httptest package, specifically
the one from the net/http/internal/testcert package:

  https://github.com/golang/go/blob/7529d09a11496a77ccbffe245607fbd256200991/src/net/http/internal/testcert/testcert.go#L10-L33

While it does include a DNSName field in its SAN list with the hostname
example.com, attempting to use this in our tests would involve forcing
both Git and Git LFS to override the available system DNS resolution.
For Git LFS in particular, there does not appear to be, at the moment,
a straightforward way of overriding the Go language's "net" package's
DNS resolution in a cross-platform and convenient manner.

Instead, with recent versions of Git and libcurl, we can bypass this
problem by setting the "http.sslBackend" Git configuration option
to "openssl", and thereby forcing the use of OpenSSL in place of
Schannel in libcurl.  This option has been available since commit
git/git@21084e8 in Git version 2.20.0,
and is supported so long as the version of libcurl is at 7.56.0.
Since our CI jobs on Windows are executed on GitHub Actions where the
pre-installed Git for Windows releases are kept current, we can
make use of this option in our tests where TLS certificate valiation
is required.

Note that this option does not affect how Git LFS behaves, because
it uses the TLS validation provided by the Go net/http and crypto/tls
packages, and because the intent of our tests is to confirm correct
behaviour of Git LFS and not Git, using OpenSSL instead of Schannel
to ensure Git accepts our test server's TLS certificate on Windows
does not compromise the integrity of our tests.  (Moreover, having
the tests actually run to completion is a big step forward over the
current state where they pass in all circumstances due to the bug
in our conditional check of the TRAVIS variable.)

As for the problem observed when the "clone ClientCert with homedir certs"
test was added, Git reports the error "refusing to work with credential
missing host field" when libcurl uses the default of the Schannel
library on Windows, and so for this reason, the test has always
been skipped on Windows.  When we enable the "clone ClientCert" test
as well by removing the check on the TRAVIS variable, it too will
report the same error.

However, we can again resolve this problem by switching to the OpenSSL
backend, as noted a discussion on the Git mailing list:

  https://lore.kernel.org/git/TY0PR06MB54426B92E745B3035F0CC2DFD197A@TY0PR06MB5442.apcprd06.prod.outlook.com/

So for both of these tests as well, we set the "http.sslBackend"
Git configuration option to "openssl" when they run on Windows, and
we can then allow the "clone ClientCert with homedir certs" to
execute on that platform for the first time since it was introduced.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 10, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the http.sslCert and http.sslKey Git configuration
values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
http.sslCert and http.sslKey Git configuration options to paths
beginning with a tilde, in order to validate our support for such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
http.sslCert and http.sslKey Git configuration options to their usual
values; that is, to the values set by the setup() shell function run
at the start of each test file.  This makes sense, because the goal
was to ensure that in all cases, the tests following the new "clone
ClientCert with homedir certs" test had the expected values for these
configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into
the home directory used for each test.  So we can simply remove
the two "git config" calls at the start of the "clone with flags" test,
as the values they set are always the same as the ones provided by
our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 15, 2024
Due to the issue described in git-lfs#5658, the "cloneSSL" and "clone ClientCert"
tests in our t/t-clone.sh test script do not run to completion but exit
early, as a consequence of an improper check of the TRAVIS variable (which
is no longer used since we migrated our test suite to GitHub Actions in
PR git-lfs#3808).

As well, since the "clone ClientCert with homedir certs" test was added
in commit c54c9da, as part of PR git-lfs#5657,
it has skipped executing on Windows due to an error that was seen when
Git attempted to retrieve the passphrase for the encrypted key of the
TLS certificate used in that test from our git-credential-lfstest
helper program.

We expect to address the first issue in subsequent commit in this PR,
at which point the tests will run fully, exposing a number of
long-standing problems.

In the case of the "cloneSSL" test, Schannel's CertGetNameString() function
will reject the TLS certificate of our lfstest-gitserver program because
we connect to the server on the local IP address 127.0.0.1, and although
the certificate lists that address as an IPAddress field among its Subject
Alternative Name (SAN) attributes, the Schannel library only looks for
hostname matches with the DNSName fields and ignores any IPAddress fields.
As a result, Git returns a "failed to match connection hostname (127.0.0.1)
against server certificate names" error, as produced by libcurl.  As of
version 8.9.0 of libcurl, the CertGetNameString() function is called
with the CERT_NAME_DNS_TYPE option, which is documented to have this
behaviour:

  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L577-L582
  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L373-L378
  https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certgetnamestringw#cert_name_dns_type

The TLS certificate presented by our lfstest-gitserver program is the
one provided by the Go language's net/http/httptest package, specifically
the one from the net/http/internal/testcert package:

  https://github.com/golang/go/blob/7529d09a11496a77ccbffe245607fbd256200991/src/net/http/internal/testcert/testcert.go#L10-L33

While it does include a DNSName field in its SAN list with the hostname
example.com, attempting to use this in our tests would involve forcing
both Git and Git LFS to override the available system DNS resolution.
For Git LFS in particular, there does not appear to be, at the moment,
a straightforward way of overriding the Go language's "net" package's
DNS resolution in a cross-platform and convenient manner.

Instead, with recent versions of Git and libcurl, we can bypass this
problem by setting the "http.sslBackend" Git configuration option
to "openssl", and thereby forcing the use of OpenSSL in place of
Schannel in libcurl.  This option has been available since commit
git/git@21084e8 in Git version 2.20.0,
and is supported so long as the version of libcurl is at 7.56.0.
Since our CI jobs on Windows are executed on GitHub Actions where the
pre-installed Git for Windows releases are kept current, we can
make use of this option in our tests where TLS certificate valiation
is required.

Note that this option does not affect how Git LFS behaves, because
it uses the TLS validation provided by the Go net/http and crypto/tls
packages, and because the intent of our tests is to confirm correct
behaviour of Git LFS and not Git, using OpenSSL instead of Schannel
to ensure Git accepts our test server's TLS certificate on Windows
does not compromise the integrity of our tests.  (Moreover, having
the tests actually run to completion is a big step forward over the
current state where they pass in all circumstances due to the bug
in our conditional check of the TRAVIS variable.)

As for the problem observed when the "clone ClientCert with homedir certs"
test was added, Git reports the error "refusing to work with credential
missing host field" when libcurl uses the default of the Schannel
library on Windows, and so for this reason, the test has always
been skipped on Windows.  When we enable the "clone ClientCert" test
as well by removing the check on the TRAVIS variable, it too will
report the same error.

However, we can again resolve this problem by switching to the OpenSSL
backend, as noted a discussion on the Git mailing list:

  https://lore.kernel.org/git/TY0PR06MB54426B92E745B3035F0CC2DFD197A@TY0PR06MB5442.apcprd06.prod.outlook.com/

So for both of these tests as well, we set the "http.sslBackend"
Git configuration option to "openssl" when they run on Windows, and
we can then allow the "clone ClientCert with homedir certs" to
execute on that platform for the first time since it was introduced.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 15, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the http.sslCert and http.sslKey Git configuration
values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
http.sslCert and http.sslKey Git configuration options to paths
beginning with a tilde, in order to validate our support for such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
http.sslCert and http.sslKey Git configuration options to their usual
values; that is, to the values set by the setup() shell function run
at the start of each test file.  This makes sense, because the goal
was to ensure that in all cases, the tests following the new "clone
ClientCert with homedir certs" test had the expected values for these
configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into
the home directory used for each test.  So we can simply remove
the two "git config" calls at the start of the "clone with flags" test,
as the values they set are always the same as the ones provided by
our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Sep 15, 2024
In commit c54c9da of PR git-lfs#5657 we add
support for the use of a client TLS certificate located in a user's home
directory, when specified using a tilde character in the path provided
for the "http.<url>.sslCert" and "http.<url>.sslKey" configuration
options.

As part of this change, we updated our setup_creds() shell test library
function to create two additional credential record files in the
directory named by our CREDSDIR environment variable.  These files
were intended to be read by our git-credential-lfstest helper program,
and contain the test passphrase for our client TLS certificate and key.

However, in practice, these files are never consulted by our test
credential helper, because the certificate and key are not encrypted and
we do not set either the "http.sslCertPasswordProtected" Git configuration
option or the GIT_SSL_CERT_PASSWORD_PROTECTED environment variable,
so neither Git nor OpenSSL search for a passphrase.

In particular, Git does not check with the configured credential helper
for a passphrase, so we don't need to provide credential record files
for our git-credential-lfstest helper program which correspond to the
filepaths of the client certificate and key files in the test user's
home directory.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 7, 2024
Due to the issue described in git-lfs#5658, the "cloneSSL" and "clone ClientCert"
tests in our t/t-clone.sh test script do not run to completion but exit
early, as a consequence of an improper check of the TRAVIS variable (which
is no longer used since we migrated our test suite to GitHub Actions in
PR git-lfs#3808).

As well, since the "clone ClientCert with homedir certs" test was added
in commit c54c9da, as part of PR git-lfs#5657,
it has skipped executing on Windows due to an error that was seen when
Git attempted to retrieve the passphrase for the encrypted key of the
TLS certificate used in that test from our git-credential-lfstest
helper program.

We expect to address the first issue in subsequent commit in this PR,
at which point the tests will run fully, exposing a number of
long-standing problems.

In the case of the "cloneSSL" test, Schannel's CertGetNameString() function
will reject the TLS certificate of our lfstest-gitserver program because
we connect to the server on the local IP address 127.0.0.1, and although
the certificate lists that address as an IPAddress field among its Subject
Alternative Name (SAN) attributes, the Schannel library only looks for
hostname matches with the DNSName fields and ignores any IPAddress fields.
As a result, Git returns a "failed to match connection hostname (127.0.0.1)
against server certificate names" error, as produced by libcurl.  As of
version 8.9.0 of libcurl, the CertGetNameString() function is called
with the CERT_NAME_DNS_TYPE option, which is documented to have this
behaviour:

  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L577-L582
  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L373-L378
  https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certgetnamestringw#cert_name_dns_type

The TLS certificate presented by our lfstest-gitserver program is the
one provided by the Go language's net/http/httptest package, specifically
the one from the net/http/internal/testcert package:

  https://github.com/golang/go/blob/7529d09a11496a77ccbffe245607fbd256200991/src/net/http/internal/testcert/testcert.go#L10-L33

While it does include a DNSName field in its SAN list with the hostname
example.com, attempting to use this in our tests would involve forcing
both Git and Git LFS to override the available system DNS resolution.
For Git LFS in particular, there does not appear to be, at the moment,
a straightforward way of overriding the Go language's "net" package's
DNS resolution in a cross-platform and convenient manner.

Instead, with recent versions of Git and libcurl, we can bypass this
problem by setting the "http.sslBackend" Git configuration option
to "openssl", and thereby forcing the use of OpenSSL in place of
Schannel in libcurl.  This option has been available since commit
git/git@21084e8 in Git version 2.20.0,
and is supported so long as the version of libcurl is at 7.56.0.
Since our CI jobs on Windows are executed on GitHub Actions where the
pre-installed Git for Windows releases are kept current, we can
make use of this option in our tests where TLS certificate valiation
is required.

Note that this option does not affect how Git LFS behaves, because
it uses the TLS validation provided by the Go net/http and crypto/tls
packages, and because the intent of our tests is to confirm correct
behaviour of Git LFS and not Git, using OpenSSL instead of Schannel
to ensure Git accepts our test server's TLS certificate on Windows
does not compromise the integrity of our tests.  (Moreover, having
the tests actually run to completion is a big step forward over the
current state where they pass in all circumstances due to the bug
in our conditional check of the TRAVIS variable.)

As for the problem observed when the "clone ClientCert with homedir certs"
test was added, Git reports the error "refusing to work with credential
missing host field" when libcurl uses the default of the Schannel
library on Windows, and so for this reason, the test has always
been skipped on Windows.  When we enable the "clone ClientCert" test
as well by removing the check on the TRAVIS variable, it too will
report the same error.

However, we can again resolve this problem by switching to the OpenSSL
backend, as noted a discussion on the Git mailing list:

  https://lore.kernel.org/git/TY0PR06MB54426B92E745B3035F0CC2DFD197A@TY0PR06MB5442.apcprd06.prod.outlook.com/

So for both of these tests as well, we set the "http.sslBackend"
Git configuration option to "openssl" when they run on Windows, and
we can then allow the "clone ClientCert with homedir certs" to
execute on that platform for the first time since it was introduced.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 7, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the http.sslCert and http.sslKey Git configuration
values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
http.sslCert and http.sslKey Git configuration options to paths
beginning with a tilde, in order to validate our support for such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
http.sslCert and http.sslKey Git configuration options to their usual
values; that is, to the values set by the setup() shell function run
at the start of each test file.  This makes sense, because the goal
was to ensure that in all cases, the tests following the new "clone
ClientCert with homedir certs" test had the expected values for these
configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into
the home directory used for each test.  So we can simply remove
the two "git config" calls at the start of the "clone with flags" test,
as the values they set are always the same as the ones provided by
our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 7, 2024
In commit c54c9da of PR git-lfs#5657 we add
support for the use of a client TLS certificate located in a user's home
directory, when specified using a tilde character in the path provided
for the "http.<url>.sslCert" and "http.<url>.sslKey" configuration
options.

As part of this change, we updated our setup_creds() shell test library
function to create two additional credential record files in the
directory named by our CREDSDIR environment variable.  These files
were intended to be read by our git-credential-lfstest helper program,
and contain the test passphrase for our client TLS certificate and key.

However, in practice, these files are never consulted by our test
credential helper, because the certificate and key are not encrypted and
we do not set either the "http.sslCertPasswordProtected" Git configuration
option or the GIT_SSL_CERT_PASSWORD_PROTECTED environment variable,
so neither Git nor OpenSSL search for a passphrase.

In particular, Git does not check with the configured credential helper
for a passphrase, so we don't need to provide credential record files
for our git-credential-lfstest helper program which correspond to the
filepaths of the client certificate and key files in the test user's
home directory.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 9, 2024
Due to the issue described in git-lfs#5658, the "cloneSSL" and "clone ClientCert"
tests in our t/t-clone.sh test script do not run to completion but exit
early, as a consequence of an improper check of the TRAVIS variable (which
is no longer used since we migrated our test suite to GitHub Actions in
PR git-lfs#3808).

As well, since the "clone ClientCert with homedir certs" test was added
in commit c54c9da, as part of PR git-lfs#5657,
we have skipped executing it on Windows.  We do this to avoid an error
when Git attempts to retrieve the encrypted key of the test's client
TLS certificate from our git-credential-lfstest test utility program.

We will address the first issue in subsequent commit in this PR, at
which point the tests will run fully, exposing a number of long-standing
problems with the "cloneSSL" and "clone ClientCert" tests.

One of these problems occurs on Windows in the "cloneSSL" test.  By
default, Git on Windows uses the Secure Channel (SChannel) backend for
TLS/SSL verification in libcurl.  However, Schannel's CertGetNameString()
function will reject the TLS certificate of our lfstest-gitserver program
because we connect to the server on the local IP address 127.0.0.1, and
although the certificate lists that address in an IPAddress field among
its Subject Alternative Name (SAN) attributes, the Schannel library only
looks for hostname matches with the DNSName fields and ignores any
IPAddress fields.  As a result, Git returns a "failed to match connection
hostname (127.0.0.1) against server certificate names" error, as produced
by libcurl.  Since version 8.9.0 of libcurl the CertGetNameString()
function is called with the CERT_NAME_DNS_TYPE option, which is
documented to have this behaviour:

  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L577-L582
  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L373-L378
  https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certgetnamestringw#cert_name_dns_type

The TLS certificate presented by our lfstest-gitserver program is the
one provided by the Go language's "net/http/httptest" package, specifically
the one from the "net/http/internal/testcert" package:

  https://github.com/golang/go/blob/7529d09a11496a77ccbffe245607fbd256200991/src/net/http/internal/testcert/testcert.go#L10-L33

While it does include a DNSName field in its SAN list with the hostname
"example.com", attempting to use this in our tests would involve forcing
both Git and Git LFS to override the available system DNS resolution.
For Git LFS in particular, there does not appear to be, at the moment,
a straightforward way of overriding the DNS resolution of the Go
language's "net" package in a cross-platform and convenient manner.

Instead, with recent versions of Git and libcurl, we can bypass this
problem by setting the "http.sslBackend" Git configuration option
to "openssl", and thereby forcing the use of OpenSSL in place of
Schannel in libcurl.  This option has been available since commit
git/git@21084e8 in Git version 2.20.0,
and is supported so long as libcurl's version is 7.56.0 or above.
Since our CI jobs on Windows are executed on GitHub Actions, where the
pre-installed Git for Windows releases are kept current, we can make
use of this Git option in our tests when TLS certificate validation
is required.

Note that this option does not affect how Git LFS behaves, because it
uses the TLS validation provided by the Go "net/http" and "crypto/tls"
packages.  Also, because the intent of our tests is to confirm correct
behaviour of Git LFS and not Git, using OpenSSL instead of Schannel
to ensure Git accepts our test server's TLS certificate on Windows
does not compromise the integrity of our tests.  (Further, having our
tests actually run to completion will be a significant improvement over
the current state, where they trivially pass in all circumstances due
to the bug in our conditional check of the TRAVIS variable.)

As for the problem mentioned earlier regarding the reason we skip the
"clone ClientCert with homedir certs" test on Windows, this also stems
from the use of the Schannel backend.  Specifically, Git reports the
error "refusing to work with credential missing host field" when
libcurl uses the Schannel library, and so we have always skipped this
test on Windows.

When we enable the "clone ClientCert" test, by removing the check on
the TRAVIS variable, it too will report the same error.

We can again resolve this problem for both tests by switching to the
OpenSSL backend, as noted in a discussion on the Git mailing list:

  https://lore.kernel.org/git/TY0PR06MB54426B92E745B3035F0CC2DFD197A@TY0PR06MB5442.apcprd06.prod.outlook.com/

So for these tests we also set the "http.sslBackend" Git configuration
option to "openssl" when executing them on Windows.  Notably, this will
allow the "clone ClientCert with homedir certs" test to run successfully
on that platform for the first time since it was introduced.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 9, 2024
Due to the issue described in git-lfs#5658, the "cloneSSL" and "clone ClientCert"
tests in our t/t-clone.sh test script do not run to completion but exit
early, as a consequence of an improper check of the TRAVIS variable (which
is no longer used since we migrated our test suite to GitHub Actions in
PR git-lfs#3808).

As well, since the "clone ClientCert with homedir certs" test was added
in commit c54c9da, as part of PR git-lfs#5657,
we have skipped executing it on Windows.  We do this to avoid an error
when Git attempts to retrieve the encrypted key of the test's client
TLS certificate from our git-credential-lfstest test utility program.

We will address the first issue in subsequent commit in this PR, at
which point the tests will run fully, exposing a number of long-standing
problems with the "cloneSSL" and "clone ClientCert" tests.

One of these problems occurs on Windows, where by default, Git uses the
Secure Channel (SChannel) backend for TLS/SSL verification in libcurl.
However, Schannel's CertGetNameString() function rejects the TLS
certificate of our lfstest-gitserver program because we connect to the
server on the local IP address 127.0.0.1, and although the certificate
lists that address in an IPAddress field among its Subject Alternative
Name (SAN) attributes, the Schannel library only looks for hostname
matches with the DNSName fields and ignores any IPAddress fields.
As a result, Git returns a "failed to match connection hostname
(127.0.0.1) against server certificate names" error, as produced by
libcurl.  Since version 8.9.0 of libcurl the CertGetNameString() function
is called with the CERT_NAME_DNS_TYPE option, which is documented to have
this behaviour:

  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L577-L582
  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L373-L378
  https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certgetnamestringw#cert_name_dns_type

The TLS certificate presented by our lfstest-gitserver program is the
one provided by the Go language's "net/http/httptest" package, specifically
the one from the "net/http/internal/testcert" package:

  https://github.com/golang/go/blob/7529d09a11496a77ccbffe245607fbd256200991/src/net/http/internal/testcert/testcert.go#L10-L33

While it does include a DNSName field in its SAN list with the hostname
"example.com", attempting to use this in our tests would involve forcing
both Git and Git LFS to override the available system DNS resolution.
For Git LFS in particular, there does not appear to be, at the moment,
a straightforward way of overriding the DNS resolution of the Go
language's "net" package in a cross-platform and convenient manner.

Instead, with recent versions of Git and libcurl, we can bypass this
problem by setting the "http.sslBackend" Git configuration option
to "openssl", and thereby forcing the use of OpenSSL in place of
Schannel in libcurl.  This option has been available since commit
git/git@21084e8 in Git version 2.20.0,
and is supported so long as libcurl's version is 7.56.0 or above.
Since our CI jobs on Windows are executed on GitHub Actions, where the
pre-installed Git for Windows releases are kept current, we can make
use of this Git option in our tests when TLS certificate validation
is required.

Note that this option does not affect how Git LFS behaves, because it
uses the TLS validation provided by the Go "net/http" and "crypto/tls"
packages.  Also, because the intent of our tests is to confirm correct
behaviour of Git LFS and not Git, using OpenSSL instead of Schannel
to ensure Git accepts our test server's TLS certificate on Windows
does not compromise the integrity of our tests.  (Further, having our
tests actually run to completion will be a significant improvement over
the current state, where they trivially pass in all circumstances due
to the bug in our conditional check of the TRAVIS variable.)

As for the problem mentioned earlier regarding the reason we skip the
"clone ClientCert with homedir certs" test on Windows, this also stems
from the use of the Schannel backend.  Specifically, Git reports the
error "refusing to work with credential missing host field" when
libcurl uses the Schannel library, and so we have always skipped this
test on Windows.

When we enable the "clone ClientCert" test, by removing the check on
the TRAVIS variable, it too will report the same error.

We can again resolve this problem for both tests by switching to the
OpenSSL backend, as noted in a discussion on the Git mailing list:

  https://lore.kernel.org/git/TY0PR06MB54426B92E745B3035F0CC2DFD197A@TY0PR06MB5442.apcprd06.prod.outlook.com/

So for these tests we also set the "http.sslBackend" Git configuration
option to "openssl" when executing them on Windows.  Notably, this will
allow the "clone ClientCert with homedir certs" test to run successfully
on that platform for the first time since it was introduced.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 29, 2024
Due to the issue described in git-lfs#5658, the "cloneSSL" and "clone ClientCert"
tests in our t/t-clone.sh test script do not run to completion but exit
early, as a consequence of an improper check of the TRAVIS variable (which
is no longer used since we migrated our test suite to GitHub Actions in
PR git-lfs#3808).

As well, since the "clone ClientCert with homedir certs" test was added
in commit c54c9da, as part of PR git-lfs#5657,
we have skipped executing it on Windows.  We do this to avoid an error
when Git attempts to retrieve the encrypted key of the test's client
TLS certificate from our git-credential-lfstest test utility program.

We will address the first issue in subsequent commit in this PR, at
which point the tests will run fully, exposing a number of long-standing
problems with the "cloneSSL" and "clone ClientCert" tests.

One of these problems occurs on Windows, where by default, Git uses the
Secure Channel (SChannel) backend for TLS/SSL verification in libcurl.
However, Schannel's CertGetNameString() function rejects the TLS
certificate of our lfstest-gitserver program because we connect to the
server on the local IP address 127.0.0.1, and although the certificate
lists that address in an IPAddress field among its Subject Alternative
Name (SAN) attributes, the Schannel library only looks for hostname
matches with the DNSName fields and ignores any IPAddress fields.
As a result, Git returns a "failed to match connection hostname
(127.0.0.1) against server certificate names" error, as produced by
libcurl.  Since version 8.9.0 of libcurl the CertGetNameString() function
is called with the CERT_NAME_DNS_TYPE option, which is documented to have
this behaviour:

  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L577-L582
  https://github.com/curl/curl/blob/5040f7e94cd01decbe7ba8fdacbf489182d503dc/lib/vtls/schannel_verify.c#L373-L378
  https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-certgetnamestringw#cert_name_dns_type

The TLS certificate presented by our lfstest-gitserver program is the
one provided by the Go language's "net/http/httptest" package, specifically
the one from the "net/http/internal/testcert" package:

  https://github.com/golang/go/blob/7529d09a11496a77ccbffe245607fbd256200991/src/net/http/internal/testcert/testcert.go#L10-L33

While it does include a DNSName field in its SAN list with the hostname
"example.com", attempting to use this in our tests would involve forcing
both Git and Git LFS to override the available system DNS resolution.
For Git LFS in particular, there does not appear to be, at the moment,
a straightforward way of overriding the DNS resolution of the Go
language's "net" package in a cross-platform and convenient manner.

Instead, with recent versions of Git and libcurl, we can bypass this
problem by setting the "http.sslBackend" Git configuration option
to "openssl", and thereby forcing the use of OpenSSL in place of
Schannel in libcurl.  This option has been available since commit
git/git@21084e8 in Git version 2.20.0,
and is supported so long as libcurl's version is 7.56.0 or above.
Since our CI jobs on Windows are executed on GitHub Actions, where the
pre-installed Git for Windows releases are kept current, we can make
use of this Git option in our tests when TLS certificate validation
is required.

Note that this option does not affect how Git LFS behaves, because it
uses the TLS validation provided by the Go "net/http" and "crypto/tls"
packages.  Also, because the intent of our tests is to confirm correct
behaviour of Git LFS and not Git, using OpenSSL instead of Schannel
to ensure Git accepts our test server's TLS certificate on Windows
does not compromise the integrity of our tests.  (Further, having our
tests actually run to completion will be a significant improvement over
the current state, where they trivially pass in all circumstances due
to the bug in our conditional check of the TRAVIS variable.)

As for the problem mentioned earlier regarding the reason we skip the
"clone ClientCert with homedir certs" test on Windows, this also stems
from the use of the Schannel backend.  Specifically, Git reports the
error "refusing to work with credential missing host field" when
libcurl uses the Schannel library, and so we have always skipped this
test on Windows.

When we enable the "clone ClientCert" test, by removing the check on
the TRAVIS variable, it too will report the same error.

We can again resolve this problem for both tests by switching to the
OpenSSL backend, as noted in a discussion on the Git mailing list:

  https://lore.kernel.org/git/TY0PR06MB54426B92E745B3035F0CC2DFD197A@TY0PR06MB5442.apcprd06.prod.outlook.com/

So for these tests we also set the "http.sslBackend" Git configuration
option to "openssl" when executing them on Windows.  Notably, this will
allow the "clone ClientCert with homedir certs" test to run successfully
on that platform for the first time since it was introduced.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 30, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the http.sslCert and http.sslKey Git configuration
values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
http.sslCert and http.sslKey Git configuration options to paths
beginning with a tilde, in order to validate our support for such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
http.sslCert and http.sslKey Git configuration options to their usual
values; that is, to the values set by the setup() shell function run
at the start of each test file.  This makes sense, because the goal
was to ensure that in all cases, the tests following the new "clone
ClientCert with homedir certs" test had the expected values for these
configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into
the home directory used for each test.  So we can simply remove
the two "git config" calls at the start of the "clone with flags" test,
as the values they set are always the same as the ones provided by
our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Oct 30, 2024
In commit c54c9da of PR git-lfs#5657 we add
support for the use of a client TLS certificate located in a user's home
directory, when specified using a tilde character in the path provided
for the "http.<url>.sslCert" and "http.<url>.sslKey" configuration
options.

As part of this change, we updated our setup_creds() shell test library
function to create two additional credential record files in the
directory named by our CREDSDIR environment variable.  These files
were intended to be read by our git-credential-lfstest helper program,
and contain the test passphrase for our client TLS certificate and key.

However, in practice, these files are never consulted by our test
credential helper, because the certificate and key are not encrypted and
we do not set either the "http.sslCertPasswordProtected" Git configuration
option or the GIT_SSL_CERT_PASSWORD_PROTECTED environment variable,
so neither Git nor OpenSSL search for a passphrase.

In particular, Git does not check with the configured credential helper
for a passphrase, so we don't need to provide credential record files
for our git-credential-lfstest helper program which correspond to the
filepaths of the client certificate and key files in the test user's
home directory.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Nov 5, 2024
In commit 9a00eb3 of PR git-lfs#1893 we
altered our "clone ClientCert" test of the Git LFS client's support
for TLS/SSL client certificates to deal with an exception raised by
the version of libcurl provided by macOS at the time.

As was reported in curl/curl#1105, the libcurl installed as part of
macOS 10.12 ("Sierra") failed to find some TLS/SSL client certificates
and instead raised an NSInvalidArgumentException.  See the notes in
our PR git-lfs#1869 for more details:

  git-lfs#1869 (comment)

To work around this issue, we skip the "clone ClientCert" test if any
NSInvalidArgumentException messages are found in the output of the Git
clone operation performed by the clone_repo_clientcert() function
of our t/testhelpers.sh shell library.

The same workaround was then copied into the "clone ClientCert with
homedir certs" test when it was added to our t/t-clone.sh test script
in commit c54c9da of PR git-lfs#5657.

The underlying problem in libcurl was resolved by the changes in commit
curl/curl@7c9b9ad, which was included
with libcurl version 7.52.0 and so is available in all contemporary
versions of macOS.  For instance, by the time of macOS 10.14 ("Mojave"),
the version of libcurl packaged with the OS was v7.54.0.

We can therefore drop our workarounds, as they are no longer applicable
to any supported versions of macOS.

(Note, too, that the GitHub Actions runners we now use for our CI jobs
provide a libcurl version installed by Homebrew, which takes precedence
over the one supplied by macOS.)
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Nov 18, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the "http.<url>.sslCert" and "http.<url>.sslKey" Git
configuration values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
"http.<url>.sslCert" and "http.<url>.sslKey" Git configuration options
to paths beginning with a tilde, in order to validate our support for
such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
"http.<url>.sslCert" and "http.<url>.sslKey" Git configuration options
to their usual values; that is, to the values set by the setup() shell
function run at the start of each test file.  This makes sense, because
the goal was to ensure that in all cases, the tests following the new
"clone ClientCert with homedir certs" test had the expected values for
these configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into the
directory used as the value of the HOME environment variable in each
test.  So we can simply remove the two "git config" calls at the start
of the "clone with flags" test, as the values they set are always the
same as the ones provided by our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Nov 18, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are prefixed
with a tilde character.  This change aligned the Git LFS client's
support of the "http.<url>.sslCert" and "http.<url>.sslKey" Git
configuration options with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
"http.<url>.sslCert" and "http.<url>.sslKey" configuration options
to paths beginning with a tilde, in order to validate our support for
such paths.

As well, the setup() and setup_creds() functions in our t/testhelpers.sh
shell library were revised to create credential record files for our
git-credential-lfstest helper program, corresponding to the paths
of the certificate file and private key file which the "clone ClientCert
with homedir certs" test copies into the location specified by the
HOME environment variable.  That location is set by our t/testlib.sh
script to a directory named "home" under the temporary directory path
stored in our TRASHDIR variable, where all our tests generate their
various artifacts.

In practice, however, because the "clone ClientCert with homedir certs"
test never sets the "http.sslCertPasswordProtected" Git configuration
option or the GIT_SSL_CERT_PASSWORD_PROTECTED environment variable,
and because it never sets the "http.<url>.sslKey" option to point to
an encrypted version of the private key file, neither Git nor Git LFS
ever invoke the "git credential fill" command.  As a result, our
credential helper program does not run, and so the record files created
by the setup_creds() function for use by this test are never opened.

Moreover, the "clone ClientCert with homedir certs" test never actually
runs the "git lfs clone" command, unlike all the other tests in the
t/t-clone.sh test script.

In previous commits in this PR we have refactored the "clone ClientCert"
test, which precedes the "clone ClientCert with homedir certs" test,
so that it more clearly distinguishes between the use of encrypted and
unencrypted private key files, and explains the conditions under which
Git and Git LFS will retrieve a passphrase for an encrypted certificate
or private key file.

We therefore copy the main loop from the "clone ClientCert" test into
the "clone ClientCert with homedir certs" test so that the latter now
runs the "git lfs clone" command twice, once using the unencrypted
version of the private key file, and once using the encrypted version.

Like the "clone ClientCert" test, the "clone ClientCert with homedir
certs" test validates the clone create by the "git lfs clone" command
on each pass through its loop, and also runs a regular "git clone"
command afterward to ensure it succeeds as well.

The test now needs to copy the encrypted private key file specified
by the LFS_CLIENT_KEY_FILE_ENCRYPTED variable into the test home
directory, since that location is then used as the value of the
"http.<url>.sslKey" configuration option during the second iteration
of the test loop.

We also slightly refactor the sequence of other initial steps performed
by the "clone ClientCert with homedir certs" test to more closely align
with the "clone ClientCert" test, such as by moving the definition of
the "reponame" variable later in the test.

We reorder the sequence in which we copy the certificate and key files
and set the "http.<url>.sslCert" and "http.<url>.sslKey" configuration
options so they follow the same sequence used in other tests and scripts.
As well, we use quotation marks around the configuration option key
names, which again aligns with our general (but not consistent) practice.

In a prior commit in this PR we added a detailed set of code comments
to the "clone ClientCert" test, which we replicate in the "clone
ClientCert with homedir certs" test.  These comments help explain the
conditions under which Git and Git LFS query or do not query the
credential helper via a "git credential fill" command, and why we need
two record files for our credential helper, both with the same
passphrase but each associated with a different file path.

Finally, because the test now establishes the record files used by the
git-credential-lfstest helper, akin to how the "clone ClientCert" test
does also, there is no need to create these record files in the common
setup_creds() function in the t/testhelpers.sh library.  Thus we can
eliminate the calls to the write_creds_file() function for these files
from the setup_creds() function, and also remove the definition of the
"homecertpath" and "homekeypath" variables from the library as well.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Nov 18, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are
prefixed with a tilde character.  This change aligned Git LFS's
support of the "http.<url>.sslCert" and "http.<url>.sslKey" Git
configuration values with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
"http.<url>.sslCert" and "http.<url>.sslKey" Git configuration options
to paths beginning with a tilde, in order to validate our support for
such paths.

As well, the following test, the "clone with flags" test, was updated
in the same commit to include a pair of commands which reset the
"http.<url>.sslCert" and "http.<url>.sslKey" Git configuration options
to their usual values; that is, to the values set by the setup() shell
function run at the start of each test file.  This makes sense, because
the goal was to ensure that in all cases, the tests following the new
"clone ClientCert with homedir certs" test had the expected values for
these configuration options.

However, resetting these values is not actually required, because
every test starts with the begin_test() shell function, which already
copies a fresh version of the global Git configuration file into the
directory used as the value of the HOME environment variable in each
test.  So we can simply remove the two "git config" calls at the start
of the "clone with flags" test, as the values they set are always the
same as the ones provided by our default test environment.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Nov 18, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are prefixed
with a tilde character.  This change aligned the Git LFS client's
support of the "http.<url>.sslCert" and "http.<url>.sslKey" Git
configuration options with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
"http.<url>.sslCert" and "http.<url>.sslKey" configuration options
to paths beginning with a tilde, in order to validate our support for
such paths.

As well, the setup() and setup_creds() functions in our t/testhelpers.sh
shell library were revised to create credential record files for our
git-credential-lfstest helper program, corresponding to the paths
of the certificate file and private key file which the "clone ClientCert
with homedir certs" test copies into the location specified by the
HOME environment variable.  That location is set by our t/testlib.sh
script to a directory named "home" under the temporary directory path
stored in our TRASHDIR variable, where all our tests generate their
various artifacts.

In practice, however, because the "clone ClientCert with homedir certs"
test never sets the "http.sslCertPasswordProtected" Git configuration
option or the GIT_SSL_CERT_PASSWORD_PROTECTED environment variable,
and because it never sets the "http.<url>.sslKey" option to point to
an encrypted version of the private key file, neither Git nor Git LFS
ever invoke the "git credential fill" command.  As a result, our
credential helper program does not run, and so the record files created
by the setup_creds() function for use by this test are never opened.

Moreover, the "clone ClientCert with homedir certs" test never actually
runs the "git lfs clone" command, unlike all the other tests in the
t/t-clone.sh test script.

In previous commits in this PR we have refactored the "clone ClientCert"
test, which precedes the "clone ClientCert with homedir certs" test,
so that it more clearly distinguishes between the use of encrypted and
unencrypted private key files, and explains the conditions under which
Git and Git LFS will retrieve a passphrase for an encrypted certificate
or private key file.

We therefore copy the main loop from the "clone ClientCert" test into
the "clone ClientCert with homedir certs" test so that the latter now
runs the "git lfs clone" command twice, once using the unencrypted
version of the private key file, and once using the encrypted version.

Like the "clone ClientCert" test, the "clone ClientCert with homedir
certs" test validates the clone create by the "git lfs clone" command
on each pass through its loop, and also runs a regular "git clone"
command afterward to ensure it succeeds as well.

The test now needs to copy the encrypted private key file specified
by the LFS_CLIENT_KEY_FILE_ENCRYPTED variable into the test home
directory, since that location is then used as the value of the
"http.<url>.sslKey" configuration option during the second iteration
of the test loop.

We also slightly refactor the sequence of other initial steps performed
by the "clone ClientCert with homedir certs" test to more closely align
with the "clone ClientCert" test, such as by moving the definition of
the "reponame" variable later in the test.

We reorder the sequence in which we copy the certificate and key files
and set the "http.<url>.sslCert" and "http.<url>.sslKey" configuration
options so they follow the same sequence used in other tests and scripts.
As well, we use quotation marks around the configuration option key
names, which again aligns with our general (but not consistent) practice.

In a prior commit in this PR we added a detailed set of code comments
to the "clone ClientCert" test, which we replicate in the "clone
ClientCert with homedir certs" test.  These comments help explain the
conditions under which Git and Git LFS query or do not query the
credential helper via a "git credential fill" command, and why we need
two record files for our credential helper, both with the same
passphrase but each associated with a different file path.

Finally, because the test now establishes the record files used by the
git-credential-lfstest helper, akin to how the "clone ClientCert" test
does also, there is no need to create these record files in the common
setup_creds() function in the t/testhelpers.sh library.  Thus we can
eliminate the calls to the write_creds_file() function for these files
from the setup_creds() function, and also remove the definition of the
"homecertpath" and "homekeypath" variables from the library as well.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Nov 18, 2024
In commit c54c9da of PR git-lfs#5657 we
added support for the use of client TLS/SSL certificate and key files
in a user's home directory, specifically, when their paths are prefixed
with a tilde character.  This change aligned the Git LFS client's
support of the "http.<url>.sslCert" and "http.<url>.sslKey" Git
configuration options with that of Git itself.

As part of this change, the "clone ClientCert with homedir certs" test
was added to our t/t-clone.sh test script.  This test sets the
"http.<url>.sslCert" and "http.<url>.sslKey" configuration options
to paths beginning with a tilde, in order to validate our support for
such paths.

As well, the setup() and setup_creds() functions in our t/testhelpers.sh
shell library were revised to create credential record files for our
git-credential-lfstest helper program, corresponding to the paths
of the certificate file and private key file which the "clone ClientCert
with homedir certs" test copies into the location specified by the
HOME environment variable.  That location is set by our t/testlib.sh
script to a directory named "home" under the temporary directory path
stored in our TRASHDIR variable, where all our tests generate their
various artifacts.

In practice, however, because the "clone ClientCert with homedir certs"
test never sets the "http.sslCertPasswordProtected" Git configuration
option or the GIT_SSL_CERT_PASSWORD_PROTECTED environment variable,
and because it never sets the "http.<url>.sslKey" option to point to
an encrypted version of the private key file, neither Git nor Git LFS
ever invoke the "git credential fill" command.  As a result, our
credential helper program does not run, and so the record files created
by the setup_creds() function for use by this test are never opened.

Moreover, the "clone ClientCert with homedir certs" test never actually
runs the "git lfs clone" command, unlike all the other tests in the
t/t-clone.sh test script.

In previous commits in this PR we have refactored the "clone ClientCert"
test, which precedes the "clone ClientCert with homedir certs" test,
so that it more clearly distinguishes between the use of encrypted and
unencrypted private key files, and explains the conditions under which
Git and Git LFS will retrieve a passphrase for an encrypted certificate
or private key file.

We therefore copy the main loop from the "clone ClientCert" test into
the "clone ClientCert with homedir certs" test so that the latter now
runs the "git lfs clone" command twice, once using the unencrypted
version of the private key file, and once using the encrypted version.

Like the "clone ClientCert" test, the "clone ClientCert with homedir
certs" test validates the clone create by the "git lfs clone" command
on each pass through its loop, and also runs a regular "git clone"
command afterward to ensure it succeeds as well.

The test now needs to copy the encrypted private key file specified
by the LFS_CLIENT_KEY_FILE_ENCRYPTED variable into the test home
directory, since that location is then used as the value of the
"http.<url>.sslKey" configuration option during the second iteration
of the test loop.

We also slightly refactor the sequence of other initial steps performed
by the "clone ClientCert with homedir certs" test to more closely align
with the "clone ClientCert" test, such as by moving the definition of
the "reponame" variable later in the test.

We reorder the sequence in which we copy the certificate and key files
and set the "http.<url>.sslCert" and "http.<url>.sslKey" configuration
options so they follow the same sequence used in other tests and scripts.
As well, we use quotation marks around the configuration option key
names, which again aligns with our general (but not consistent) practice.

In a prior commit in this PR we added a detailed set of code comments
to the "clone ClientCert" test, which we replicate in the "clone
ClientCert with homedir certs" test.  These comments help explain the
conditions under which Git and Git LFS query or do not query the
credential helper via a "git credential fill" command, and why we need
two record files for our credential helper, both with the same
passphrase but each associated with a different file path.

Finally, because the test now establishes the record files used by the
git-credential-lfstest helper, akin to how the "clone ClientCert" test
does also, there is no need to create these record files in the common
setup_creds() function in the t/testhelpers.sh library.  Thus we can
eliminate the calls to the write_creds_file() function for these files
from the setup_creds() function, and also remove the definition of the
"homecertpath" and "homekeypath" variables from the library as well.

Note that when we call the write_creds_file() function to create the
credential record files we need to use paths which align with what our
git-credential-lfstest helper will construct from the values it receives
in the "path" fields of its input.  On Windows these "path" fields will
contain Windows-style paths with a leading volume identifier like "D:".

As discussed in commit bc11a31 of
PR git-lfs#5882, our git-credential-lfstest helper converts such paths so
they begin with a lowercase letter preceded by a slash character,
like "/d", in order to match the paths found in the variables used
to create the filenames of the credential record files in our "clone
ClientCert" test.  In our CI jobs, which run in the Git Bash environment,
which uses the MSYS2 environment, these LFS_CLIENT_CERT_FILE and
LFS_CLIENT_KEY_FILE_ENCRYPTED variables contain paths of the "mixed"
Cygwin form, so they have Unix-style slash characters but start with
a leading volume identifier segment such as "/d".

In the "clone ClientCert with homedir certs" test, because we do not
use these variables to construct the filenames of the credential
record files, but the HOME environment variable instead, we have to
perform the same path translation using the "cygwin" program, and
then converting its output so volume identifiers like "D:" are
converted into the "/d" form.  This ensures the record files we
create have the same names as the git-credential-lfstest helper
will construct.  Note, too, that because the colon character is
disallowed in Windows filenames (other than in a volume identifier),
we can not simply use the "D:" form of the paths when constructing the
record file names, so this technique resolves that problem as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Client certificates silently ignored if paths specified with "~/"

2 participants