Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

pks-t
Copy link
Member

@pks-t pks-t commented Oct 12, 2018

This cherry-picks all pull requests currently labelled "backport-0.27.6". Most notable is probably the change to our CI infrastructure. Please have a look and let me know if any commits should be dropped or if I missed some importants PRs that should be part of this.

@pks-t
Copy link
Member Author

pks-t commented Oct 12, 2018

I've redone this PR. Instead of pulling in #4798, I decided to use the backports script to cherry-pick all relevant CI PRs. This is because #4798 was missing the XML clar changes, and backporting these was near impossible due to dozens of conflicts. With the backporting script it was a lot more manageable. @ethomson, please let me know if that's okay for you.

I've also spotted some out-of-date backports in #4798, which have been fixed by the reroll.

Diff to v1:

diff --git a/README.md b/README.md
index d297aaa10..1ec0bd800 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
 libgit2 - the Git linkable library
 ==================================
 
-[![VSTS Build Status](https://libgit2.visualstudio.com/libgit2/_apis/build/status/libgit2)](https://libgit2.visualstudio.com/libgit2/_build/latest?definitionId=7)
+[![Azure Pipelines Build Status](https://dev.azure.com/libgit2/libgit2/_apis/build/status/libgit2)](https://dev.azure.com/libgit2/libgit2/_build/latest?definitionId=7)
 [![Coverity Scan Build Status](https://scan.coverity.com/projects/639/badge.svg)](https://scan.coverity.com/projects/639)
 
 `libgit2` is a portable, pure C implementation of the Git core methods
@@ -17,7 +17,7 @@ in your favorite language.
 [GitKraken](https://gitkraken.com/) and [gmaster](https://gmaster.io/)
 and on Git hosting providers like [GitHub](https://github.com/),
 [GitLab](https://gitlab.com/) and
-[Visual Studio Team Services](https://visualstudio.com/team-services/).
+[Azure DevOps](https://azure.com/devops).
 We perform the merge every time you click "merge pull request".
 
 `libgit2` is licensed under a **very permissive license** (GPLv2 with a special
@@ -63,7 +63,7 @@ slack channel once you've registered.
 
 If you have questions about the library, please be sure to check out the
 [API documentation](http://libgit2.github.com/libgit2/).  If you still have
-questions, reach out to us on Slack or post a question on
+questions, reach out to us on Slack or post a question on 
 [StackOverflow](http://stackoverflow.com/questions/tagged/libgit2) (with the `libgit2` tag).
 
 **Reporting Bugs**
@@ -86,7 +86,7 @@ What It Can Do
 
 libgit2 provides you with the ability to manage Git repositories in the
 programming language of your choice.  It's used in production to power many
-applications including GitHub.com, Plastic SCM and Visual Studio Team Services.
+applications including GitHub.com, Plastic SCM and Azure DevOps.
 
 It does not aim to replace the git tool or its user-facing commands. Some APIs
 resemble the plumbing commands as those align closely with the concepts of the
diff --git a/azure-pipelines.yml b/azure-pipelines.yml
index 6d9374d72..64007d1a2 100644
--- a/azure-pipelines.yml
+++ b/azure-pipelines.yml
@@ -29,9 +29,6 @@ jobs:
       environmentVariables: |
        CC=clang
        LEAK_CHECK=valgrind
-      workDir: '/build'
-      containerCommand: '/src/ci/test.sh'
-      detached: false
 
 - job: macos
   displayName: 'macOS'
diff --git a/ci/build.sh b/ci/build.sh
index a1deab3f2..3757be48d 100755
--- a/ci/build.sh
+++ b/ci/build.sh
@@ -28,8 +28,8 @@ echo "##########################################################################
 echo "## Configuring build environment"
 echo "##############################################################################"
 
-echo cmake ${SOURCE_DIR} -DBUILD_EXAMPLES=ON ${CMAKE_OPTIONS}
-cmake ${SOURCE_DIR} -DBUILD_EXAMPLES=ON ${CMAKE_OPTIONS}
+echo cmake ${SOURCE_DIR} -DENABLE_WERROR=ON -DBUILD_EXAMPLES=ON ${CMAKE_OPTIONS}
+cmake ${SOURCE_DIR} -DENABLE_WERROR=ON -DBUILD_EXAMPLES=ON ${CMAKE_OPTIONS}
 
 echo ""
 echo "##############################################################################"
diff --git a/ci/setup-linux.sh b/ci/setup-linux.sh
index c5ecb550b..a0db14ee0 100755
--- a/ci/setup-linux.sh
+++ b/ci/setup-linux.sh
@@ -11,17 +11,3 @@ if [ -z "$SKIP_APT" ]; then
 fi
 
 mkdir -p /var/run/sshd
-
-if [ "$MBEDTLS" ]; then
-	MBEDTLS_DIR=${MBEDTLS_DIR:-$(mktemp -d ${TMPDIR}/mbedtls.XXXXXXXX)}
-
-	git clone --depth 10 --single-branch --branch mbedtls-2.6.1 https://github.com/ARMmbed/mbedtls.git ${MBEDTLS_DIR}
-	cd ${MBEDTLS_DIR}
-
-	CFLAGS=-fPIC cmake -DENABLE_PROGRAMS=OFF -DENABLE_TESTING=OFF -DUSE_SHARED_MBEDTLS_LIBRARY=OFF -DUSE_STATIC_MBEDTLS_LIBRARY=ON .
-	cmake --build .
-
-	if [ -z "$SKIP_MBEDTLS_INSTALL" ]; then
-		make install
-	fi
-fi
diff --git a/ci/test.ps1 b/ci/test.ps1
index 21e8bd764..7a55bff79 100644
--- a/ci/test.ps1
+++ b/ci/test.ps1
@@ -5,38 +5,68 @@ $PSDefaultParameterValues['*:ErrorAction'] = 'Stop'
 
 [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
 
+$SourceDir = Split-Path (Split-Path (Get-Variable MyInvocation).Value.MyCommand.Path)
+$BuildDir = Get-Location
+$Success = $true
+
 if ($Env:SKIP_TESTS) { exit }
 
+# Ask ctest what it would run if we were to invoke it directly.  This lets
+# us manage the test configuration in a single place (tests/CMakeLists.txt)
+# instead of running clar here as well.  But it allows us to wrap our test
+# harness with a leak checker like valgrind.  Append the option to write
+# JUnit-style XML files.
+function run_test {
+	$TestName = $args[0]
+
+	$TestCommand = (ctest -N -V -R "^$TestName$") -join "`n" -replace "(?ms).*\n^[0-9]*: Test command: ","" -replace "\n.*",""
+	$TestCommand += " -r${BuildDir}\results_${TestName}.xml"
+
+	Write-Host $TestCommand
+	Invoke-Expression $TestCommand
+
+	if ($LastExitCode -ne 0) { $Success = $false }
+}
+
 Write-Host "##############################################################################"
 Write-Host "## Configuring test environment"
 Write-Host "##############################################################################"
 
-Write-Host ""
-Write-Host "Starting HTTP proxy..."
-Invoke-WebRequest -Method GET -Uri https://github.com/ethomson/poxyproxy/releases/download/v0.1.0/poxyproxy-0.1.0.jar -OutFile poxyproxy.jar
-javaw -jar poxyproxy.jar -d --port 8080 --credentials foo:bar
+if (-not $Env:SKIP_PROXY_TESTS) {
+	Write-Host ""
+	Write-Host "Starting HTTP proxy..."
+	Invoke-WebRequest -Method GET -Uri https://github.com/ethomson/poxyproxy/releases/download/v0.1.0/poxyproxy-0.1.0.jar -OutFile poxyproxy.jar
+	javaw -jar poxyproxy.jar -d --port 8080 --credentials foo:bar
+}
 
 Write-Host ""
 Write-Host "##############################################################################"
-Write-Host "## Running default tests"
+Write-Host "## Running (offline) tests"
 Write-Host "##############################################################################"
 
-ctest -V -R libgit2_clar
-if ($LastExitCode -ne 0) { [Environment]::Exit($LastExitCode) }
+run_test offline
 
-Write-Host ""
-Write-Host "##############################################################################"
-Write-Host "## Running online tests"
-Write-Host "##############################################################################"
+if (-not $Env:SKIP_ONLINE_TESTS) {
+	Write-Host ""
+	Write-Host "##############################################################################"
+	Write-Host "## Running (online) tests"
+	Write-Host "##############################################################################"
 
-Write-Host ""
-Write-Host "Running proxy tests"
-Write-Host ""
+	run_test online
+}
+
+if (-not $Env:SKIP_PROXY_TESTS) {
+	Write-Host ""
+	Write-Host "Running proxy tests"
+	Write-Host ""
+
+	$Env:GITTEST_REMOTE_PROXY_URL="localhost:8080"
+	$Env:GITTEST_REMOTE_PROXY_USER="foo"
+	$Env:GITTEST_REMOTE_PROXY_PASS="bar"
+
+	run_test proxy
 
-$Env:GITTEST_REMOTE_PROXY_URL="localhost:8080"
-$Env:GITTEST_REMOTE_PROXY_USER="foo"
-$Env:GITTEST_REMOTE_PROXY_PASS="bar"
-ctest -V -R libgit2_clar-proxy_credentials
-if ($LastExitCode -ne 0) { [Environment]::Exit($LastExitCode) }
+	taskkill /F /IM javaw.exe
+}
 
-taskkill /F /IM javaw.exe
+if (-not $Success) { exit 1 }
diff --git a/ci/test.sh b/ci/test.sh
index e9736461f..fea9d82d7 100755
--- a/ci/test.sh
+++ b/ci/test.sh
@@ -11,6 +11,8 @@ BUILD_DIR=$(pwd)
 TMPDIR=${TMPDIR:-/tmp}
 USER=${USER:-$(whoami)}
 
+SUCCESS=1
+
 VALGRIND="valgrind --leak-check=full --show-reachable=yes --error-exitcode=125 --num-callers=50 --suppressions=\"$SOURCE_DIR/libgit2_clar.supp\""
 LEAKS="MallocStackLogging=1 MallocScribble=1 leaks -quiet -atExit -- nohup"
 
@@ -30,18 +32,19 @@ cleanup() {
 	echo "Done."
 }
 
-die() {
+failure() {
 	echo "Test exited with code: $1"
-
-	cleanup
-	exit $1
+	SUCCESS=0
 }
 
-# Ask ctest what it would run if we were to invoke it directly.  This lets us manage the
-# test configuration in a single place (tests/CMakeLists.txt) instead of running clar
-# here as well.  But it allows us to wrap our test harness with a leak checker like valgrind.
+# Ask ctest what it would run if we were to invoke it directly.  This lets
+# us manage the test configuration in a single place (tests/CMakeLists.txt)
+# instead of running clar here as well.  But it allows us to wrap our test
+# harness with a leak checker like valgrind.  Append the option to write
+# JUnit-style XML files.
 run_test() {
 	TEST_CMD=$(ctest -N -V -R "^${1}$" | sed -n 's/^[0-9]*: Test command: //p')
+	TEST_CMD="${TEST_CMD} -r${BUILD_DIR}/results_${1}.xml"
 
 	if [ "$LEAK_CHECK" = "valgrind" ]; then
 		RUNNER="$VALGRIND $TEST_CMD"
@@ -51,7 +54,7 @@ run_test() {
 		RUNNER="$TEST_CMD"
 	fi
 
-	eval $RUNNER || die $?
+	eval $RUNNER || failure
 }
 
 # Configure the test environment; run them early so that we're certain
@@ -118,26 +121,38 @@ fi
 
 # Run the tests that do not require network connectivity.
 
+if [ -z "$SKIP_OFFLINE_TESTS" ]; then
+	echo ""
+	echo "##############################################################################"
+	echo "## Running (offline) tests"
+	echo "##############################################################################"
+
+	run_test offline
+fi
+
+if [ -z "$SKIP_ONLINE_TESTS" ]; then
+	# Run the various online tests.  The "online" test suite only includes the
+	# default online tests that do not require additional configuration.  The
+	# "proxy" and "ssh" test suites require further setup.
+
+	echo ""
+	echo "##############################################################################"
+	echo "## Running (online) tests"
+	echo "##############################################################################"
+
+	run_test online
+fi
+
 if [ -z "$SKIP_GITDAEMON_TESTS" ]; then
+	echo ""
+	echo "Running gitdaemon tests"
+	echo ""
+
 	export GITTEST_REMOTE_URL="git://localhost/test.git"
+	run_test gitdaemon
+	unset GITTEST_REMOTE_URL
 fi
 
-echo ""
-echo "##############################################################################"
-echo "## Running (offline) tests"
-echo "##############################################################################"
-
-run_test libgit2_clar
-
-# Run the various online tests.  The "online" test suite only includes the
-# default online tests that do not require additional configuration.  The
-# "proxy" and "ssh" test suites require further setup.
-
-echo ""
-echo "##############################################################################"
-echo "## Running (online) tests"
-echo "##############################################################################"
-
 if [ -z "$SKIP_PROXY_TESTS" ]; then
 	echo ""
 	echo "Running proxy tests"
@@ -146,7 +161,7 @@ if [ -z "$SKIP_PROXY_TESTS" ]; then
 	export GITTEST_REMOTE_PROXY_URL="localhost:8080"
 	export GITTEST_REMOTE_PROXY_USER="foo"
 	export GITTEST_REMOTE_PROXY_PASS="bar"
-	run_test libgit2_clar-proxy_credentials
+	run_test proxy
 	unset GITTEST_REMOTE_PROXY_URL
 	unset GITTEST_REMOTE_PROXY_USER
 	unset GITTEST_REMOTE_PROXY_PASS
@@ -163,7 +178,7 @@ if [ -z "$SKIP_SSH_TESTS" ]; then
 	export GITTEST_REMOTE_SSH_PUBKEY="${HOME}/.ssh/id_rsa.pub"
 	export GITTEST_REMOTE_SSH_PASSPHRASE=""
 	export GITTEST_REMOTE_SSH_FINGERPRINT="${SSH_FINGERPRINT}"
-	run_test libgit2_clar-ssh
+	run_test ssh
 	unset GITTEST_REMOTE_URL
 	unset GITTEST_REMOTE_USER
 	unset GITTEST_REMOTE_SSH_KEY
@@ -172,6 +187,12 @@ if [ -z "$SKIP_SSH_TESTS" ]; then
 	unset GITTEST_REMOTE_SSH_FINGERPRINT
 fi
 
-echo "Success."
 cleanup
+
+if [ "$SUCCESS" -ne "1" ]; then
+	echo "Some tests failed."
+	exit 1
+fi
+
+echo "Success."
 exit 0
diff --git a/src/odb.c b/src/odb.c
index ef9c87555..ede2aa57b 100644
--- a/src/odb.c
+++ b/src/odb.c
@@ -1384,8 +1384,8 @@ static int git_odb_stream__invalid_length(
 {
 	giterr_set(GITERR_ODB,
 		"cannot %s - "
-		"Invalid length. %"PRIdZ" was expected. The "
-		"total size of the received chunks amounts to %"PRIdZ".",
+		"Invalid length. %"PRId64" was expected. The "
+		"total size of the received chunks amounts to %"PRId64".",
 		action, stream->declared_size, stream->received_bytes);
 
 	return -1;
diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt
index 82bf6d0d7..02042df6d 100644
--- a/tests/CMakeLists.txt
+++ b/tests/CMakeLists.txt
@@ -52,12 +52,8 @@ IF (MSVC_IDE)
 	SET_SOURCE_FILES_PROPERTIES("precompiled.c" COMPILE_FLAGS "/Ycprecompiled.h")
 ENDIF ()
 
-IF (USE_HTTPS)
-	ADD_TEST(libgit2_clar "${libgit2_BINARY_DIR}/libgit2_clar" -ionline -xclone::local::git_style_unc_paths -xclone::local::standard_unc_paths_are_written_git_style)
-ELSE ()
-	ADD_TEST(libgit2_clar "${libgit2_BINARY_DIR}/libgit2_clar" -v -xclone::local::git_style_unc_paths -xclone::local::standard_unc_paths_are_written_git_style)
-ENDIF ()
-
-# Add additional test targets that require special setup
-ADD_TEST(libgit2_clar-proxy_credentials "${libgit2_BINARY_DIR}/libgit2_clar" -v -sonline::clone::proxy_credentials_in_url -sonline::clone::proxy_credentials_request)
-ADD_TEST(libgit2_clar-ssh "${libgit2_BINARY_DIR}/libgit2_clar" -v -sonline::push -sonline::clone::ssh_cert -sonline::clone::ssh_with_paths)
+ADD_TEST(offline   "${libgit2_BINARY_DIR}/libgit2_clar" -v -xonline)
+ADD_TEST(online    "${libgit2_BINARY_DIR}/libgit2_clar" -v -sonline)
+ADD_TEST(gitdaemon "${libgit2_BINARY_DIR}/libgit2_clar" -v -sonline::push)
+ADD_TEST(ssh       "${libgit2_BINARY_DIR}/libgit2_clar" -v -sonline::push -sonline::clone::ssh_cert -sonline::clone::ssh_with_paths)
+ADD_TEST(proxy     "${libgit2_BINARY_DIR}/libgit2_clar" -v -sonline::clone::proxy_credentials_in_url -sonline::clone::proxy_credentials_request)
diff --git a/tests/clar.c b/tests/clar.c
index d5212d1ca..27d35e1c7 100644
--- a/tests/clar.c
+++ b/tests/clar.c
@@ -95,9 +95,6 @@ static const char *
 fixture_path(const char *base, const char *fixture_name);
 
 struct clar_error {
-	const char *test;
-	int test_number;
-	const char *suite;
 	const char *file;
 	int line_number;
 	const char *error_msg;
@@ -106,11 +103,34 @@ struct clar_error {
 	struct clar_error *next;
 };
 
+struct clar_explicit {
+	size_t suite_idx;
+	const char *filter;
+
+	struct clar_explicit *next;
+};
+
+struct clar_report {
+	const char *test;
+	int test_number;
+	const char *suite;
+
+	enum cl_test_status status;
+
+	struct clar_error *errors;
+	struct clar_error *last_error;
+
+	struct clar_report *next;
+};
+
+struct clar_summary {
+	const char *filename;
+	FILE *fp;
+};
+
 static struct {
-	int argc;
-	char **argv;
-
 	enum cl_test_status test_status;
+
 	const char *active_test;
 	const char *active_suite;
 
@@ -124,8 +144,15 @@ static struct {
 	int exit_on_error;
 	int report_suite_names;
 
-	struct clar_error *errors;
-	struct clar_error *last_error;
+	int write_summary;
+	const char *summary_filename;
+	struct clar_summary *summary;
+
+	struct clar_explicit *explicit;
+	struct clar_explicit *last_explicit;
+
+	struct clar_report *reports;
+	struct clar_report *last_report;
 
 	void (*local_cleanup)(void *);
 	void *local_cleanup_payload;
@@ -155,7 +182,7 @@ struct clar_suite {
 /* From clar_print_*.c */
 static void clar_print_init(int test_count, int suite_count, const char *suite_names);
 static void clar_print_shutdown(int test_count, int suite_count, int error_count);
-static void clar_print_error(int num, const struct clar_error *error);
+static void clar_print_error(int num, const struct clar_report *report, const struct clar_error *error);
 static void clar_print_ontest(const char *test_name, int test_number, enum cl_test_status failed);
 static void clar_print_onsuite(const char *suite_name, int suite_index);
 static void clar_print_onabort(const char *msg, ...);
@@ -164,6 +191,10 @@ static void clar_print_onabort(const char *msg, ...);
 static void clar_unsandbox(void);
 static int clar_sandbox(void);
 
+/* From summary.h */
+static struct clar_summary *clar_summary_init(const char *filename);
+static int clar_summary_shutdown(struct clar_summary *fp);
+
 /* Load the declarations for the test suite */
 #include "clar.suite"
 
@@ -186,21 +217,29 @@ void cl_trace_register(cl_trace_cb *cb, void *payload)
 
 /* Core test functions */
 static void
-clar_report_errors(void)
+clar_report_errors(struct clar_report *report)
 {
+	struct clar_error *error;
 	int i = 1;
-	struct clar_error *error, *next;
-
-	error = _clar.errors;
-	while (error != NULL) {
-		next = error->next;
-		clar_print_error(i++, error);
-		free(error->description);
-		free(error);
-		error = next;
+
+	for (error = report->errors; error; error = error->next)
+		clar_print_error(i++, _clar.last_report, error);
+}
+
+static void
+clar_report_all(void)
+{
+	struct clar_report *report;
+	struct clar_error *error;
+	int i = 1;
+
+	for (report = _clar.reports; report; report = report->next) {
+		if (report->status != CL_TEST_FAILURE)
+			continue;
+
+		for (error = report->errors; error; error = error->next)
+			clar_print_error(i++, report, error);
 	}
-
-	_clar.errors = _clar.last_error = NULL;
 }
 
 static void
@@ -209,7 +248,6 @@ clar_run_test(
 	const struct clar_func *initialize,
 	const struct clar_func *cleanup)
 {
-	_clar.test_status = CL_TEST_OK;
 	_clar.trampoline_enabled = 1;
 
 	CL_TRACE(CL_TRACE__TEST__BEGIN);
@@ -225,6 +263,9 @@ clar_run_test(
 
 	_clar.trampoline_enabled = 0;
 
+	if (_clar.last_report->status == CL_TEST_NOTRUN)
+		_clar.last_report->status = CL_TEST_OK;
+
 	if (_clar.local_cleanup != NULL)
 		_clar.local_cleanup(_clar.local_cleanup_payload);
 
@@ -240,9 +281,9 @@ clar_run_test(
 	_clar.local_cleanup_payload = NULL;
 
 	if (_clar.report_errors_only) {
-		clar_report_errors();
+		clar_report_errors(_clar.last_report);
 	} else {
-		clar_print_ontest(test->name, _clar.tests_ran, _clar.test_status);
+		clar_print_ontest(test->name, _clar.tests_ran, _clar.last_report->status);
 	}
 }
 
@@ -251,6 +292,7 @@ clar_run_suite(const struct clar_suite *suite, const char *filter)
 {
 	const struct clar_func *test = suite->tests;
 	size_t i, matchlen;
+	struct clar_report *report;
 
 	if (!suite->enabled)
 		return;
@@ -283,6 +325,21 @@ clar_run_suite(const struct clar_suite *suite, const char *filter)
 			continue;
 
 		_clar.active_test = test[i].name;
+
+		report = calloc(1, sizeof(struct clar_report));
+		report->suite = _clar.active_suite;
+		report->test = _clar.active_test;
+		report->test_number = _clar.tests_ran;
+		report->status = CL_TEST_NOTRUN;
+
+		if (_clar.reports == NULL)
+			_clar.reports = report;
+
+		if (_clar.last_report != NULL)
+			_clar.last_report->next = report;
+
+		_clar.last_report = report;
+
 		clar_run_test(&test[i], &suite->initialize, &suite->cleanup);
 
 		if (_clar.exit_on_error && _clar.total_errors)
@@ -298,13 +355,14 @@ clar_usage(const char *arg)
 {
 	printf("Usage: %s [options]\n\n", arg);
 	printf("Options:\n");
-	printf("  -sname\tRun only the suite with `name` (can go to individual test name)\n");
-	printf("  -iname\tInclude the suite with `name`\n");
-	printf("  -xname\tExclude the suite with `name`\n");
-	printf("  -v    \tIncrease verbosity (show suite names)\n");
-	printf("  -q    \tOnly report tests that had an error\n");
-	printf("  -Q    \tQuit as soon as a test fails\n");
-	printf("  -l    \tPrint suite names\n");
+	printf("  -sname        Run only the suite with `name` (can go to individual test name)\n");
+	printf("  -iname        Include the suite with `name`\n");
+	printf("  -xname        Exclude the suite with `name`\n");
+	printf("  -v            Increase verbosity (show suite names)\n");
+	printf("  -q            Only report tests that had an error\n");
+	printf("  -Q            Quit as soon as a test fails\n");
+	printf("  -l            Print suite names\n");
+	printf("  -r[filename]  Write summary file (to the optional filename)\n");
 	exit(-1);
 }
 
@@ -318,7 +376,7 @@ clar_parse_args(int argc, char **argv)
 		char *argument = argv[i];
 
 		if (argument[0] != '-' || argument[1] == '\0'
-		    || strchr("sixvqQl", argument[1]) == NULL) {
+		    || strchr("sixvqQlr", argument[1]) == NULL) {
 			clar_usage(argv[0]);
 		}
 	}
@@ -359,7 +417,24 @@ clar_parse_args(int argc, char **argv)
 						_clar.report_suite_names = 1;
 
 					switch (action) {
-					case 's': _clar_suites[j].enabled = 1; clar_run_suite(&_clar_suites[j], argument); break;
+					case 's': {
+						struct clar_explicit *explicit =
+							calloc(1, sizeof(struct clar_explicit));
+						assert(explicit);
+
+						explicit->suite_idx = j;
+						explicit->filter = argument;
+
+						if (_clar.explicit == NULL)
+							_clar.explicit = explicit;
+
+						if (_clar.last_explicit != NULL)
+							_clar.last_explicit->next = explicit;
+
+						_clar_suites[j].enabled = 1;
+						_clar.last_explicit = explicit;
+						break;
+					}
 					case 'i': _clar_suites[j].enabled = 1; break;
 					case 'x': _clar_suites[j].enabled = 0; break;
 					}
@@ -397,6 +472,12 @@ clar_parse_args(int argc, char **argv)
 			_clar.report_suite_names = 1;
 			break;
 
+		case 'r':
+			_clar.write_summary = 1;
+			_clar.summary_filename = *(argument + 2) ? (argument + 2) :
+			    "summary.xml";
+			break;
+
 		default:
 			assert(!"Unexpected commandline argument!");
 		}
@@ -412,23 +493,31 @@ clar_test_init(int argc, char **argv)
 		""
 	);
 
+	if (argc > 1)
+		clar_parse_args(argc, argv);
+
+	if (_clar.write_summary &&
+	    !(_clar.summary = clar_summary_init(_clar.summary_filename))) {
+		clar_print_onabort("Failed to open the summary file\n");
+		exit(-1);
+	}
+
 	if (clar_sandbox() < 0) {
 		clar_print_onabort("Failed to sandbox the test runner.\n");
 		exit(-1);
 	}
-
-	_clar.argc = argc;
-	_clar.argv = argv;
 }
 
 int
 clar_test_run(void)
 {
-	if (_clar.argc > 1)
-		clar_parse_args(_clar.argc, _clar.argv);
+	size_t i;
+	struct clar_explicit *explicit;
 
-	if (!_clar.suites_ran) {
-		size_t i;
+	if (_clar.explicit) {
+		for (explicit = _clar.explicit; explicit; explicit = explicit->next)
+			clar_run_suite(&_clar_suites[explicit->suite_idx], explicit->filter);
+	} else {
 		for (i = 0; i < _clar_suite_count; ++i)
 			clar_run_suite(&_clar_suites[i], NULL);
 	}
@@ -439,6 +528,9 @@ clar_test_run(void)
 void
 clar_test_shutdown(void)
 {
+	struct clar_explicit *explicit, *explicit_next;
+	struct clar_report *report, *report_next;
+
 	clar_print_shutdown(
 		_clar.tests_ran,
 		(int)_clar_suite_count,
@@ -446,6 +538,21 @@ clar_test_shutdown(void)
 	);
 
 	clar_unsandbox();
+
+	if (_clar.write_summary && clar_summary_shutdown(_clar.summary) < 0) {
+		clar_print_onabort("Failed to write the summary file\n");
+		exit(-1);
+	}
+
+	for (explicit = _clar.explicit; explicit; explicit = explicit_next) {
+		explicit_next = explicit->next;
+		free(explicit);
+	}
+
+	for (report = _clar.reports; report; report = report_next) {
+		report_next = report->next;
+		free(report);
+	}
 }
 
 int
@@ -465,7 +572,7 @@ static void abort_test(void)
 	if (!_clar.trampoline_enabled) {
 		clar_print_onabort(
 				"Fatal error: a cleanup method raised an exception.");
-		clar_report_errors();
+		clar_report_errors(_clar.last_report);
 		exit(-1);
 	}
 
@@ -475,7 +582,7 @@ static void abort_test(void)
 
 void clar__skip(void)
 {
-	_clar.test_status = CL_TEST_SKIP;
+	_clar.last_report->status = CL_TEST_SKIP;
 	_clar.total_skipped++;
 	abort_test();
 }
@@ -489,17 +596,14 @@ void clar__fail(
 {
 	struct clar_error *error = calloc(1, sizeof(struct clar_error));
 
-	if (_clar.errors == NULL)
-		_clar.errors = error;
+	if (_clar.last_report->errors == NULL)
+		_clar.last_report->errors = error;
 
-	if (_clar.last_error != NULL)
-		_clar.last_error->next = error;
+	if (_clar.last_report->last_error != NULL)
+		_clar.last_report->last_error->next = error;
 
-	_clar.last_error = error;
+	_clar.last_report->last_error = error;
 
-	error->test = _clar.active_test;
-	error->test_number = _clar.tests_ran;
-	error->suite = _clar.active_suite;
 	error->file = file;
 	error->line_number = line;
 	error->error_msg = error_msg;
@@ -508,7 +612,7 @@ void clar__fail(
 		error->description = strdup(description);
 
 	_clar.total_errors++;
-	_clar.test_status = CL_TEST_FAILURE;
+	_clar.last_report->status = CL_TEST_FAILURE;
 
 	if (should_abort)
 		abort_test();
@@ -653,3 +757,4 @@ void cl_set_cleanup(void (*cleanup)(void *), void *opaque)
 #include "clar/fixtures.h"
 #include "clar/fs.h"
 #include "clar/print.h"
+#include "clar/summary.h"
diff --git a/tests/clar.h b/tests/clar.h
index 5c674d70f..bdaab09d7 100644
--- a/tests/clar.h
+++ b/tests/clar.h
@@ -12,13 +12,16 @@
 enum cl_test_status {
 	CL_TEST_OK,
 	CL_TEST_FAILURE,
-	CL_TEST_SKIP
+	CL_TEST_SKIP,
+	CL_TEST_NOTRUN,
 };
 
+/** Setup clar environment */
 void clar_test_init(int argc, char *argv[]);
 int clar_test_run(void);
 void clar_test_shutdown(void);
 
+/** One shot setup & run */
 int clar_test(int argc, char *argv[]);
 
 const char *clar_sandbox_path(void);
diff --git a/tests/clar/print.h b/tests/clar/print.h
index 916d807c1..73c4a8953 100644
--- a/tests/clar/print.h
+++ b/tests/clar/print.h
@@ -13,16 +13,16 @@ static void clar_print_shutdown(int test_count, int suite_count, int error_count
 	(void)error_count;
 
 	printf("\n\n");
-	clar_report_errors();
+	clar_report_all();
 }
 
-static void clar_print_error(int num, const struct clar_error *error)
+static void clar_print_error(int num, const struct clar_report *report, const struct clar_error *error)
 {
 	printf("  %d) Failure:\n", num);
 
 	printf("%s::%s [%s:%d]\n",
-		error->suite,
-		error->test,
+		report->suite,
+		report->test,
 		error->file,
 		error->line_number);
 
@@ -44,6 +44,7 @@ static void clar_print_ontest(const char *test_name, int test_number, enum cl_te
 	case CL_TEST_OK: printf("."); break;
 	case CL_TEST_FAILURE: printf("F"); break;
 	case CL_TEST_SKIP: printf("S"); break;
+	case CL_TEST_NOTRUN: printf("N"); break;
 	}
 
 	fflush(stdout);
diff --git a/tests/clar/summary.h b/tests/clar/summary.h
new file mode 100644
index 000000000..1af110efa
--- /dev/null
+++ b/tests/clar/summary.h
@@ -0,0 +1,134 @@
+
+#include <stdio.h>
+#include <time.h>
+
+int clar_summary_close_tag(
+    struct clar_summary *summary, const char *tag, int indent)
+{
+	const char *indt;
+
+	if (indent == 0) indt = "";
+	else if (indent == 1) indt = "\t";
+	else indt = "\t\t";
+
+	return fprintf(summary->fp, "%s</%s>\n", indt, tag);
+}
+
+int clar_summary_testsuites(struct clar_summary *summary)
+{
+	return fprintf(summary->fp, "<testsuites>\n");
+}
+
+int clar_summary_testsuite(struct clar_summary *summary,
+    int idn, const char *name, const char *pkg, time_t timestamp,
+    double elapsed, int test_count, int fail_count, int error_count)
+{
+	struct tm *tm = localtime(&timestamp);
+	char iso_dt[20];
+
+	if (strftime(iso_dt, sizeof(iso_dt), "%Y-%m-%dT%H:%M:%S", tm) == 0)
+		return -1;
+
+	return fprintf(summary->fp, "\t<testsuite "
+		       " id=\"%d\""
+		       " name=\"%s\""
+		       " package=\"%s\""
+		       " hostname=\"localhost\""
+		       " timestamp=\"%s\""
+		       " time=\"%.2f\""
+		       " tests=\"%d\""
+		       " failures=\"%d\""
+		       " errors=\"%d\">\n",
+		       idn, name, pkg, iso_dt, elapsed, test_count, fail_count, error_count);
+}
+
+int clar_summary_testcase(struct clar_summary *summary,
+    const char *name, const char *classname, double elapsed)
+{
+	return fprintf(summary->fp,
+	    "\t\t<testcase name=\"%s\" classname=\"%s\" time=\"%.2f\">\n",
+		name, classname, elapsed);
+}
+
+int clar_summary_failure(struct clar_summary *summary,
+    const char *type, const char *message, const char *desc)
+{
+	return fprintf(summary->fp,
+	    "\t\t\t<failure type=\"%s\"><![CDATA[%s\n%s]]></failure>\n",
+	    type, message, desc);
+}
+
+struct clar_summary *clar_summary_init(const char *filename)
+{
+	struct clar_summary *summary;
+	FILE *fp;
+
+	if ((fp = fopen(filename, "w")) == NULL)
+		return NULL;
+
+	if ((summary = malloc(sizeof(struct clar_summary))) == NULL) {
+		fclose(fp);
+		return NULL;
+	}
+
+	summary->filename = filename;
+	summary->fp = fp;
+
+	return summary;
+}
+
+int clar_summary_shutdown(struct clar_summary *summary)
+{
+	struct clar_report *report;
+	const char *last_suite = NULL;
+
+	if (clar_summary_testsuites(summary) < 0)
+		goto on_error;
+
+	report = _clar.reports;
+	while (report != NULL) {
+		struct clar_error *error = report->errors;
+
+		if (last_suite == NULL || strcmp(last_suite, report->suite) != 0) {
+			if (clar_summary_testsuite(summary, 0, report->suite, "",
+			    time(NULL), 0, _clar.tests_ran, _clar.total_errors, 0) < 0)
+				goto on_error;
+		}
+
+		last_suite = report->suite;
+
+		clar_summary_testcase(summary, report->test, "what", 0);
+
+		while (error != NULL) {
+			if (clar_summary_failure(summary, "assert",
+			    error->error_msg, error->description) < 0)
+				goto on_error;
+
+			error = error->next;
+		}
+
+		if (clar_summary_close_tag(summary, "testcase", 2) < 0)
+			goto on_error;
+
+		report = report->next;
+
+		if (!report || strcmp(last_suite, report->suite) != 0) {
+			if (clar_summary_close_tag(summary, "testsuite", 1) < 0)
+				goto on_error;
+		}
+	}
+
+	if (clar_summary_close_tag(summary, "testsuites", 0) < 0 ||
+	    fclose(summary->fp) != 0)
+		goto on_error;
+
+	printf("written summary file to %s\n", summary->filename);
+
+	free(summary);
+	return 0;
+
+on_error:
+	fclose(summary->fp);
+	free(summary);
+	return -1;
+}
diff --git a/tests/online/clone.c b/tests/online/clone.c
index 4d409cb77..3fc6ddd03 100644
--- a/tests/online/clone.c
+++ b/tests/online/clone.c
@@ -263,8 +263,8 @@ static int cred_failure_cb(
 
 void test_online_clone__cred_callback_failure_return_code_is_tunnelled(void)
 {
-	git__free(_remote_url); _remote_url = NULL;
-	git__free(_remote_user); _remote_user = NULL;
+	git__free(_remote_url);
+	git__free(_remote_user);
 
 	_remote_url = git__strdup("https://github.com/libgit2/non-existent");
 	_remote_user = git__strdup("libgit2test");
@@ -296,8 +296,8 @@ void test_online_clone__cred_callback_called_again_on_auth_failure(void)
 {
 	size_t counter = 0;
 
-	git__free(_remote_url); _remote_url = NULL;
-	git__free(_remote_user); _remote_user = NULL;
+	git__free(_remote_url);
+	git__free(_remote_user);
 
 	_remote_url = git__strdup("https://github.com/libgit2/non-existent");
 	_remote_user = git__strdup("libgit2test");

@ethomson
Copy link
Member

I very intentionally didn't include the XML changes to clar. Changing the way our test runner works in a rather fundamental way like that seemed very risky for a point release, that's why I had done all this work manually.

@pks-t
Copy link
Member Author

pks-t commented Oct 12, 2018

I thought it might be something like that, but to be honest I don't quite get the risk,. First, the new XML stuff is optional and only in place with "-r". Sure, it might be the new code actually broke something even without that switch. Second, it's only our tests and not some kind of API. But even if people have scripted against it e.g. in packaging instructions, it would still work just fine as the new XML mode is optional after all. And without the XML stuff, the CI is only half-functioning (well, at least not optimally functioning) as it cannot deduce any information about the tests.

@ethomson
Copy link
Member

Sure - I don't think it's a big risk, but we changed the way we buffer (or don't) test results, even when you're not using the XML output. But at this point we've been using it on the master branch for a while now, so it's probably a reasonable enough risk.

@pks-t
Copy link
Member Author

pks-t commented Oct 12, 2018

Fair enough. I'm fine with taking this risk.

That being said, I'd also be fine with reverting to the old state of this PR using #4798 if you wish, especially as you're the one who has done most of the migration to VSTS.

@ethomson
Copy link
Member

🤷‍♂️

@ethomson
Copy link
Member

I think we should probably just merge this. The important thing for me is getting the new CI scripts merged so that we can set up all the platforms on our point releases. 👍

@pks-t pks-t changed the title Bugix release v0.27.6 Bugix release v0.27.7 Oct 26, 2018
tiennou and others added 21 commits October 26, 2018 14:58
We were previously conflating any error into GIT_ENOTFOUND, which might
or might not be correct. This fixes the code so a config error is
bubbled up, as well as preserving the semantics in the face of
worktree-repositories
When we add entries to a treebuilder we validate them. But we validate even
those that we're adding because they exist in the base tree. This disables
using the normal mechanisms on these trees, even to fix them.

Keep track of whether the entry we're appending comes from an existing tree and
bypass the name and id validation if it's from existing data.

(cherry picked from commit 2dff7e2)
String operations in libgit2 are supposed to never receive `NULL`, e.g.
they are not `NULL`-save. In the case of `git__linenlen()`, invocation
with `NULL` leads to undefined behavior.

In a `git_parse_ctx` however, the `content` field used in these
operations was initialized to `NULL` if the `git_parse_ctx_init()` was
called with `NULL` for `content` or `0` for `content_len`. For the
latter case, the initialization function even contained some logic for
initializing `content` with `NULL`.

This commit mitigates triggering undefined behavior by rewriting the
logic. Now `content` is always initialized to a non-null buffer. Instead
of a null buffer, an empty string is used for denoting an empty buffer.

(cherry picked from commit d1bfe61)
(cherry picked from commit ec76a1a)
(cherry picked from commit f140950)
(cherry picked from commit 6698e05)
The function `git_diff_find_similar` keeps a function of cache
similarity metrics signatures, whose size depends on the number of
deltas passed in via the `diff` parameter. In case where the diff is
empty and thus doesn't have any deltas at all, we may end up allocating
this cache via a call to `git__calloc(0, sizeof(void *))`. At least on
AIX, allocating 0 bytes will result in a `NULL` pointer being returned,
which causes us to erroneously return an OOM error.

Fix this situation by simply returning early in case where we are being
passed an empty diff, as we cannot find any similarities in that case
anyway.

(cherry picked from commit c65568d)
This performs a compile-check by using CMake support, to differentiate the GNU
version from the BSD version of qsort_r.

Module taken from 4f252abea5f1d17c60f6ff115c9c44cc0b6f1df6, which I've checked
against CMake 2.8.11.

(cherry picked from commit 1a9cc18)
(cherry picked from commit 581d549)
The documentation states that git_worktree_unlock returns 0 on success,
and 1 on success if the worktree wasn't locked. Turns out we were
returning 0 in any of those cases.

(cherry picked from commit 59c2e70)
At line 594, we do this :
if (error < 0)
		return error;

but if nothing was pushed in a GIT_SORT_TIME revwalk, we'd return
uninitialized stack data.

(cherry picked from commit aa8cb58)
Otherwise we'll return stack data to the caller.

(cherry picked from commit 22d013b)
While most systems provide a separate iconv library against which
applications can link, musl based systems do not provide such a library.
Instead, iconv functions are directly included in the C library. As our
current CMake module to locate the iconv library only checks whether a
library exists somewhere in the typical library directories, we will
never build libgit2 with libiconv support on such systems.

Extend the iconv module to also search whether libc provides iconv
functions, which we do by checking whether the `iconv_open` function
exists inside of libc. If this is the case, we will default to use the
libc provided one instead of trying to use a separate libiconv. While
this changes which iconv we use on systems where both libc and an
external libiconv exist, to the best of my knowledge common systems only
provide either one or the other.

Note that libiconv support in musl is held kind of basic. To quote musl
libc's page on functional differences from glibc [1]:

    The iconv implementation musl is very small and oriented towards
    being unobtrusive to static link. Its character set/encoding
    coverage is very strong for its size, but not comprehensive like
    glibc’s.

As we assume iconv to be a lot more capable than what musl provides,
some of our tests will fail if using iconv on musl-based platforms.

[1]: https://wiki.musl-libc.org/functional-differences-from-glibc.html

(cherry picked from commit 2e2d8c6)
Previously we would assert in index_free because the reader incrementation
would not be balanced. Release the snapshot normally, so the variable gets
decremented before the index is freed.

(cherry picked from commit c70713d)
carlosmn and others added 10 commits October 26, 2018 14:58
When porting, we overlooked that the difference between git's and our's time
representation and copied their way of getting the max value.

Unfortunately git was using unsigned integers, so `~0ll` does correspond to
their max value, whereas for us it corresponds to `-1`. This means that we
always consider the last date to be smaller than the current commit's and always
think commits are interesting.

Change the initial value to the macro that gives us the maximum value on each
platform so we can accurately consider commits interesting or not.

(cherry picked from commit 46f3512)
…tamp

This is not a big deal, but it does make us match git more closely by checking
only the first. The lists are sorted already, so there should be no functional
difference other than removing a possible check from every iteration in the
loop.

(cherry picked from commit 12a1790)
Blobs that have been marked as uninteresting should not be inserted into packbuilder
when inserting a tree. The check as to whether a blob was uninteresting looked at
the status for the tree itself instead of the blob.

This could cause significantly larger packfiles.

(cherry picked from commit b36cc7a)
We do not currently have any warnings in this regard, but it's good practice to
have them on in case we introduce something.

(cherry picked from commit f2c1153)
GCC warns by default when implicitly converting integers to pointers or
the other way round, and commit fa48d2e (vector: do not malloc
0-length vectors on dup, 2018-09-26) introduced such an implicit
conversion into our vector tests. While this is totally fine in this
test, as the pointer's value is never being used in the first place, we
can trivially avoid the warning by instead just inserting a pointer for
a variable allocated on the stack into the vector.

(cherry picked from commit dbb4a58)
While GCC enables int-conversion warnings by default, it will currently
only warn about such errors even in case where "-DENABLE_WERROR=ON" has
been passed to CMake. Explicitly enable int-conversion warnings by using
our `ENABLE_WARNINGS` macro, which will automatically use
"-Werror=int-conversions" in case it has been requested by the user.

(cherry picked from commit aa0ae59)
Otherwise we return a NULL context, which will get dereferenced in 
apply_credentials.

(cherry picked from commit 1c949ce)
auth_context_match returns 0 instead of -1 for unknown schemes to
not fail in situations where some authentication schemes are supported
and others are not.

apply_credentials is adjusted to handle auth_context_match returning
0 without producing authentication context.

(cherry picked from commit 475db39)
(cherry picked from commit bfec652)
Early Windows TLS 1.2 implementations have an issue during key exchange
with OpenSSL implementations that cause negotiation to fail with the
error "the buffer supplied to a function was too small."

This is a transient error on the connection, so when that error is
received, retry up to 5 times to create a connection to the remote
server before actually giving up.

(cherry picked from commit dc371e3)
@pks-t pks-t merged commit f23dc5b into libgit2:maint/v0.27 Oct 26, 2018
@pks-t pks-t deleted the pks/v0.27.6 branch October 26, 2018 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants