libFuzzer: Add build support and instructions #4433

lhchavez · 2017-12-07T06:20:11Z

This change adds support for building a fuzz target for exercising the
packfile parser, as well as documentation. It also runs the fuzzers in
Travis to avoid regressions.

lhchavez · 2017-12-07T06:24:32Z

PTAL!

Note that this depends on merging the following three changes, to avoid crashing wildly:

The OSS-Fuzz support will come next. Stay tuned!

pks-t

Thanks a lot for this. I have no idea how @carlosmn or @ethomson think about this, but eventually I'd like to have a set of fuzzers included in our code base. It would be even nicer if those fuzzers could be submitted to the Google OSS-fuzz infrastructure such that we have continuous fuzzing of some of our code. I'd deem this especially important for all network-facing code.

So thanks a lot for working on this, this crosses off one item from my todo list!

pks-t · 2017-12-08T10:49:37Z

CMakeLists.txt

@@ -328,6 +329,10 @@ IF (BUILD_EXAMPLES)
 	ADD_SUBDIRECTORY(examples)
 ENDIF ()

+IF (BUILD_FUZZERS)


Even though that's not yet documented, we now settled on IF( style, so without spaces in between. We're not being consistent about it right now, but going forward we try to be.

pks-t · 2017-12-08T10:52:50Z

fuzz/README.md

+
+1. Create a build directory beneath the libgit2 source directory, and change into it: `mkdir build && cd build`
+2. Choose which sanitizers to add. For instance, `address,leak`. Note that some sanitizers are mutually incompatible, so you might need several rounds to achieve more coverage.
+3. Create the cmake build environment and configure the build with the sanitizers chosen: `CC=/usr/bin/clang-6.0 CFLAGS="-fsanitize=fuzzer-no-link,address,leak -g" cmake -DTHREADSAFE=ON -DBUILD_CLAR=OFF -DBUILD_SHARED_LIBS=OFF -DBUILD_FUZZERS=ON -DCMAKE_BUILD_TYPE="RelWithDebInfo" -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=fuzzer,address,leak" ..`. Note that `CFLAGS` has `fuzzer-no-link` followed by the chosen sanitizers and `-DCMAKE_EXE_LINKER_FLAGS` has `fuzzer` followed by the chosen sanitizers.


I think instead of instructing the user to set CC and CFLAGS for himself, you should do it for him in case the user has enabled fuzzing.

Tried to automate it as much as possible, but I had trouble getting rid of the CC and linker flags, because OSS-Fuzz require the build system to not assume these things.

I could add an USE_OSS_FUZZ flag to turn off all these things, but I'm not an expert in CMake and couldn't get it to correctly detect a version of clang that was recent enough (especially given that you need to manually install it, and the binary name might change between distros).

It's a lot better already. Given the clear build instructions I'm fine with this, thanks!

pks-t · 2017-12-08T10:53:40Z

fuzz/README.md

+
+1. All the prerequisites for [building libgit2](https://github.com/libgit2/libgit2).
+2. A recent version of clang. 6.0 is preferred. [pre-build Debian/Ubuntu packages](https://github.com/libgit2/libgit2)
+3. (Optional) Symlink the `llvm-symbolizer` binary to get nice stack traces: `sudo ln -s llvm-symbolizer-6.0 /usr/bin/llvm-symbolizer`


We shouldn't instruct the user to modify his base system, which should only be touched by the package manager.

pks-t · 2017-12-08T10:55:57Z

fuzz/README.md

+
+**Run the fuzz targets**
+
+1. `ASAN_OPTIONS=allocator_may_return_null=1 ./build/fuzz/fuzz_packfile_raw fuzz/corpora/fuzz_packfile_raw/`


What's that "corpora" prefix?

The plural of "corpus": http://llvm.org/docs/LibFuzzer.html#corpus

Heh, good to know

pks-t · 2017-12-08T10:58:36Z

fuzz/fuzz_packfile_raw.c

+#include <stdint.h>
+#include <stdio.h>
+
+#include <openssl/sha.h>


Instead of using OpenSSL for SHA1, you can just link to libgit2internal, add the include path to the source directory and use for example git_hash_buf to do the hashing.

Good idea, done

pks-t · 2017-12-08T11:00:02Z

fuzz/fuzz_packfile_raw.c

+static git_odb *odb = NULL;
+
+int LLVMFuzzerInitialize(__attribute__((unused)) int *argc,
+			 __attribute__((unused)) char ***argv)


Please use the GIT_UNUSED macro instead of __attribute((unused)).

pks-t · 2017-12-08T11:04:55Z

fuzz/fuzz_packfile_raw.c

+		fprintf(stderr, "Failed to initialize libgit2\n");
+		abort();
+	}
+	if (git_repository_init(&repo, REPOSITORY_PATH, true) < 0) {


I think instead of having a repository being created in the current directory, you should instead create a temporary path by using e.g. mkdtemp.

How about I use git_mempack_new instead? No files except the pack_git_... temporaries which will only be leaked if the process crashes. I also added some code to remove the *.pack and *.idx files that are committed.

The problem with the fuzzers is that they are not designed to clean up the process state when they go down. Even using atexit(3) wasn't enough to clean the mess left.

That's even better!

pks-t · 2017-12-08T11:05:40Z

fuzz/fuzz_packfile_raw.c

+	}
+
+	git_indexer *indexer;
+	git_transfer_progress stats = {0, 0};


We are using C90 only, so please no variable declarations after statements.

Done (switching my brain off of C++ is hard)

pks-t · 2017-12-08T11:08:00Z

fuzz/fuzz_packfile_raw.c

+			abort();
+		}
+		if (git_indexer_append(indexer, hash, sizeof(hash), &stats) <
+		    0) {


This linebreak is a bit funny. No need to be too strict with breaking lines.

I admit my kernel-style is weak, so I rely on clang-format. Fixed.

pks-t · 2017-12-08T11:09:14Z

fuzz/fuzz_packfile_raw.c

+
+cleanup:
+	git_indexer_free(indexer);
+	return 0;


How is the return value being interpreted by LLVM?

It is required to be zero (for now): http://llvm.org/docs/LibFuzzer.html#fuzz-target

lhchavez · 2017-12-09T21:59:40Z

The trigger is ready to be pulled: https://github.com/lhchavez/oss-fuzz/tree/libgit2/projects/libgit2

LMK who needs to be the owner/maintainer for the project in OSS-Fuzz.

lhchavez · 2017-12-23T17:06:04Z

All prerequisites have been merged, so this change is ready to land.

pks-t · 2018-01-03T18:41:40Z

I think from our side I'm the person the most interested in fuzzing, so that might be a natural choice. Any opinions on this @ethomson or @carlosmn? I'll also do a review tomorrow

pks-t

I didn't see anything that immediately catched my eyes, only some few things to clear up for myself. I'd welcome it though if you could rebase this PR to get rid of the merge and also try to squash commits together where it makes sense (not in the sense of having one giant commit, but in the sense of getting rid of fixups to just-introduced commits).

Before merging this I'd like to get the opinions of @ethomson and @carlosmn.

Thanks a lot for your work!

pks-t · 2018-01-04T12:10:13Z

fuzz/fuzz_packfile_raw.c

+		fprintf(stderr, "Failed to initialize libgit2\n");
+		abort();
+	}
+	if (git_repository_init(&repo, REPOSITORY_PATH, true) < 0) {


That's even better!

pks-t · 2018-01-04T12:12:50Z

fuzz/fuzz_packfile_raw.c

- * with this software. If not, see
- * <http://creativecommons.org/publicdomain/zero/1.0/>.
+ * This file is part of libgit2, distributed under the GNU GPL v2 with
+ * a Linking Exception. For full terms see the included COPYING file.


This should definitly be squashed into the commit which created that file ;)

pks-t · 2018-01-04T12:14:12Z

fuzz/README.md

+
+1. Create a build directory beneath the libgit2 source directory, and change into it: `mkdir build && cd build`
+2. Choose which sanitizers to add. For instance, `address,leak`. Note that some sanitizers are mutually incompatible, so you might need several rounds to achieve more coverage.
+3. Create the cmake build environment and configure the build with the sanitizers chosen: `CC=/usr/bin/clang-6.0 CFLAGS="-fsanitize=fuzzer-no-link,address,leak -g" cmake -DTHREADSAFE=ON -DBUILD_CLAR=OFF -DBUILD_SHARED_LIBS=OFF -DBUILD_FUZZERS=ON -DCMAKE_BUILD_TYPE="RelWithDebInfo" -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=fuzzer,address,leak" ..`. Note that `CFLAGS` has `fuzzer-no-link` followed by the chosen sanitizers and `-DCMAKE_EXE_LINKER_FLAGS` has `fuzzer` followed by the chosen sanitizers.


It's a lot better already. Given the clear build instructions I'm fine with this, thanks!

pks-t · 2018-01-04T12:14:39Z

CMakeLists.txt

+	# The actual sanitizer link target will be added when linking the fuzz
+	# targets.
+	SET(CMAKE_C_FLAGS "-fsanitize=fuzzer-no-link ${CMAKE_C_FLAGS}")
+ENDIF ()


pks-t · 2018-01-04T12:16:51Z

fuzz/README.md

+
+In order to get full coverage, though, you might want to also enable one of the
+sanitizers. You might need a recent version of clang to get full support.
+


That's definitly nice to have. I'm currently stuck on a system without working fuzzers due to problems with musl libc, so I always have to use a VM. At least I can get some feedback with the standalone mode.

pks-t · 2018-01-04T12:18:50Z

fuzz/StandaloneFuzzTargetMain.c

+		goto exit;
+	}
+	git_vector_foreach(&corpus_files, i, corpus_filename) {
+		if (run_one_file(corpus_filename) < 0) {


This can only happen in case the corpus file could not be read, right? As in the case of a corpus error, we alwasy abort().

yes, abort() or segfault.

pks-t · 2018-01-04T12:20:31Z

script/cibuild.sh

+	fuzz_target_name=$(basename "${fuzz_target}")
+	# TODO: Make this exit on failures once all fixes are in place.
+	"${fuzz_target}" "../fuzz/corpora/${fuzz_target_name}"
+done


Cool, that's a good thing to have. I guess making it work on AppVeyor wouldn't make any sense at all, would it?

maybe, i can give it a try later :)

lhchavez · 2018-01-04T15:44:09Z

Squashed and ready to go!

pks-t · 2018-02-09T10:54:05Z

I'll tackle this PR after the next release, which should hopefully be done soon

This change adds support for building a fuzz target for exercising the packfile parser, as well as documentation. It also runs the fuzzers in Travis to avoid regressions.

lhchavez · 2018-07-07T23:05:58Z

any progress on the release? i've rebased the squashed change to HEAD.

pks-t · 2018-07-13T08:06:16Z

Sorry @lhchavez for not getting back to this earlier x( I've planned to tackle this issue next week anyway, as we now also have another proposed fuzzer from @nelhage. As both proposed fuzzers test different things (packfiles vs smart protocol), they luckily can both be merged into our code base.

This PR has been lurking around for too long, though. I always wanted to get this merged, but somehow I never found the time to finally follow through.

lhchavez · 2018-07-14T13:55:05Z

all right, let me know if there's a preferred way to split this change to make reviews easier.

pks-t · 2018-07-19T08:13:17Z

Would you be okay with me adopting this PR? I'd obviously retain your commits and start working from them, but adopting the PR would allow me to push this faster and also merge with the other fuzzing target we have by now. Feel free to say "no" in case you want to push this through, though, I wouldn't mind.

lhchavez · 2018-07-19T12:49:48Z

go for it! and thanks for your help.

ethomson · 2018-10-20T19:11:06Z

@pks-t You've included all these changes in #4728, correct? Since that's been merged, this PR is now obsolete and can be closed, is that correct?

pks-t · 2018-11-02T07:05:53Z

Correct, fuzzing is implemented now and finding lots of errors, so this can be closed Thanks again, @lhchavez, it has proven to be highly valuable for us!

pks-t reviewed Dec 8, 2017

View reviewed changes

pks-t reviewed Jan 4, 2018

View reviewed changes

lhchavez force-pushed the fuzzing branch from 76fbe7b to 78f5426 Compare January 4, 2018 15:43

This was referenced Jun 25, 2018

Fix assorted leaks found via fuzzing #4698

Merged

oss-fuzz integration for libgit2 #4697

Closed

libFuzzer: Add build support and instructions

9e5d333

This change adds support for building a fuzz target for exercising the packfile parser, as well as documentation. It also runs the fuzzers in Travis to avoid regressions.

lhchavez force-pushed the fuzzing branch from 78f5426 to 9e5d333 Compare July 7, 2018 21:53

pks-t mentioned this pull request Jul 19, 2018

Fuzzers #4728

Merged

pks-t closed this Nov 2, 2018


		Run the fuzz targets

		1. `ASAN_OPTIONS=allocator_may_return_null=1 ./build/fuzz/fuzz_packfile_raw fuzz/corpora/fuzz_packfile_raw/`


		In order to get full coverage, though, you might want to also enable one of the
		sanitizers. You might need a recent version of clang to get full support.

libFuzzer: Add build support and instructions #4433

libFuzzer: Add build support and instructions #4433

Uh oh!

Conversation

lhchavez commented Dec 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lhchavez commented Dec 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pks-t left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lhchavez commented Dec 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lhchavez commented Dec 23, 2017

Uh oh!

pks-t commented Jan 3, 2018

Uh oh!

pks-t left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

lhchavez commented Dec 7, 2017 •

edited

Loading

lhchavez commented Dec 7, 2017 •

edited

Loading

lhchavez commented Dec 9, 2017 •

edited

Loading