Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit e64e3b8

Browse files
codexByron
codex
and
Byron
committed
Add corpus-builder scripts when corpus files are available; auto-run artifacts in test suite
Co-authored-by: Sebastian Thiel <[email protected]>
1 parent 0396152 commit e64e3b8

26 files changed

Lines changed: 294 additions & 59 deletions

File tree

‎etc/fuzz-corpus-builder.sh‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Shared helper for libFuzzer corpus builder scripts.
2+
#
3+
# Usage from a fuzz target-specific builder:
4+
#
5+
# root="$(readlink -f -- "$1")"
6+
# output_corpus="$2"
7+
# source "$root/etc/fuzz-corpus-builder.sh"
8+
# build_fuzz_corpus "$root" "$output_corpus" "gix-pack" "index_file"
9+
#
10+
# The target-specific script is invoked by Google OSS Fuzz and
11+
# their build script at https://github.com/google/oss-fuzz/blob/master/projects/gitoxide/build.sh
12+
# with the repository root and the output zip path.
13+
# This helper adds all files from:
14+
#
15+
# <crate>/fuzz/corpus/<target>/*
16+
#
17+
# If present, it also adds files from:
18+
#
19+
# <crate>/fuzz/artifacts/<target>/*
20+
#
21+
# Corpus files and artifact files are zipped with `-j`, matching the flat corpus
22+
# archive layout it expects. Empty input groups are skipped so artifact-only
23+
# targets can still produce a corpus archive. Artifacts are added in a separate
24+
# zip invocation so a duplicate basename updates the archive instead of making
25+
# zip fail with "cannot repeat names in zip file".
26+
build_fuzz_corpus() {
27+
local root="$1"
28+
local output_corpus="$2"
29+
local crate="$3"
30+
local target="$4"
31+
local fuzz_dir
32+
local corpus_dir
33+
local artifacts_dir
34+
35+
fuzz_dir="$(readlink -f -- "$root/$crate/fuzz")"
36+
corpus_dir="$fuzz_dir/corpus/$target"
37+
artifacts_dir="$fuzz_dir/artifacts/$target"
38+
39+
echo "$root"
40+
echo "$corpus_dir"
41+
echo "$artifacts_dir"
42+
43+
shopt -s nullglob
44+
45+
local corpus_files=("$corpus_dir"/*)
46+
if ((${#corpus_files[@]})); then
47+
zip -j "$output_corpus" "${corpus_files[@]}"
48+
fi
49+
50+
if [[ -d "$artifacts_dir" ]]; then
51+
local artifact_files=("$artifacts_dir"/*)
52+
if ((${#artifact_files[@]})); then
53+
# A second zip invocation updates duplicate basenames instead of aborting.
54+
zip -j "$output_corpus" "${artifact_files[@]}"
55+
fi
56+
fi
57+
}

‎gix-actor/fuzz/fuzz_targets/actors_corpus_builder.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
3+
set -eux -o pipefail
4+
5+
root="$(readlink -f -- "$1")"
6+
output_corpus="$2"
7+
8+
source "$root/etc/fuzz-corpus-builder.sh"
9+
10+
build_fuzz_corpus "$root" "$output_corpus" "gix-actor" "actors"

‎gix-bitmap/fuzz/README.md‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Fuzz Testing
2+
3+
This directory contains fuzz tests for `gix-bitmap` using [cargo-fuzz](https://rust-fuzz.github.io/book/cargo-fuzz.html).
4+
5+
## Running Fuzz Tests
6+
7+
### Prerequisites
8+
- Nightly Rust toolchain: `rustup install nightly`
9+
- cargo-fuzz: `cargo install cargo-fuzz`
10+
11+
### Targets
12+
- `ewah`: exercises EWAH bitmap decoding.
13+
14+
### Running the target
15+
```bash
16+
cargo +nightly fuzz run ewah -- -max_total_time=60
17+
```
18+
19+
## Artifact Reproducers
20+
21+
The `artifacts/ewah` directory contains minimized reproducer inputs produced by libFuzzer or Google OSS-Fuzz.
22+
23+
The integration test module `fuzzed` reads every regular file in that directory and runs it through the parser under test. This lets `cargo test -p gix-bitmap fuzzed` reproduce known fuzz failures without starting a fuzzing session.
24+
25+
When OSS-Fuzz reports a new failure, place the minimized testcase in `artifacts/ewah`, then run `cargo test -p gix-bitmap fuzzed`. To confirm it against the original harness, run `cargo fuzz run ewah artifacts/ewah/<reproducer>` from this directory.

‎gix-bitmap/fuzz/fuzz_targets/ewah_corpus_builder.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
3+
set -eux -o pipefail
4+
5+
root="$(readlink -f -- "$1")"
6+
output_corpus="$2"
7+
8+
source "$root/etc/fuzz-corpus-builder.sh"
9+
10+
build_fuzz_corpus "$root" "$output_corpus" "gix-bitmap" "ewah"

‎gix-bitmap/tests/bitmap.rs‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,12 @@
11
mod fuzzed {
2+
#[test]
3+
fn ewah_artifacts_run_fuzzer() {
4+
for path in artifact_paths("ewah") {
5+
let data = std::fs::read(path).expect("artifact is readable");
6+
let _ = gix_bitmap::ewah::decode(&data);
7+
}
8+
}
9+
210
#[test]
311
fn runaway_run_length_is_rejected() {
412
let (bitmap, rest) = gix_bitmap::ewah::decode(include_bytes!(
@@ -13,4 +21,18 @@ mod fuzzed {
1321
"impossible run lengths must be rejected instead of iterating unboundedly"
1422
);
1523
}
24+
25+
fn artifact_paths(target: &str) -> Vec<std::path::PathBuf> {
26+
std::fs::read_dir(
27+
std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"))
28+
.join("fuzz/artifacts")
29+
.join(target),
30+
)
31+
.ok()
32+
.into_iter()
33+
.flatten()
34+
.filter_map(Result::ok)
35+
.map(|entry| entry.path())
36+
.collect()
37+
}
1638
}

‎gix-command/fuzz/fuzz_targets/prepare_corpus_builder.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
3+
set -eux -o pipefail
4+
5+
root="$(readlink -f -- "$1")"
6+
output_corpus="$2"
7+
8+
source "$root/etc/fuzz-corpus-builder.sh"
9+
10+
build_fuzz_corpus "$root" "$output_corpus" "gix-command" "prepare"

‎gix-config/fuzz/fuzz_targets/fuzz_file_corpus_builder.sh‎

Lines changed: 0 additions & 22 deletions
This file was deleted.

‎gix-credentials/fuzz/fuzz_targets/context_corpus_builder.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
3+
set -eux -o pipefail
4+
5+
root="$(readlink -f -- "$1")"
6+
output_corpus="$2"
7+
8+
source "$root/etc/fuzz-corpus-builder.sh"
9+
10+
build_fuzz_corpus "$root" "$output_corpus" "gix-credentials" "context"

‎gix-ignore/fuzz/fuzz_targets/ignore_corpus_builder.sh‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
#!/usr/bin/env bash
2+
3+
set -eux -o pipefail
4+
5+
root="$(readlink -f -- "$1")"
6+
output_corpus="$2"
7+
8+
source "$root/etc/fuzz-corpus-builder.sh"
9+
10+
build_fuzz_corpus "$root" "$output_corpus" "gix-ignore" "ignore"

‎gix-index/fuzz/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,11 @@ This directory contains fuzz tests for `gix-index` using [cargo-fuzz](https://ru
1515
```bash
1616
cargo +nightly fuzz run index_file -- -max_total_time=60
1717
```
18+
19+
## Artifact Reproducers
20+
21+
The `artifacts/index_file` directory contains minimized reproducer inputs produced by libFuzzer or Google OSS-Fuzz.
22+
23+
The integration test module `fuzzed` reads every regular file in that directory and runs it through the parser under test. This lets `cargo test -p gix-index-tests fuzzed` reproduce known fuzz failures without starting a fuzzing session.
24+
25+
When OSS-Fuzz reports a new failure, place the minimized testcase in `artifacts/index_file`, then run `cargo test -p gix-index-tests fuzzed`. To confirm it against the original harness, run `cargo fuzz run index_file artifacts/index_file/<reproducer>` from this directory.

0 commit comments

Comments
 (0)