BugOSS is a benchmark of real-world regression bugs found in OSS-Fuzz for experimenting with regression fuzzing techniques. To reproduce the real project context where the bugs were introduced, each study artifact of BugOSS indicates the exact bug-inducing commit, and provides the information about the target bug, together with the existing bugs in the same commit. Currently, 20 artifacts from 20 C/C++ programs are registered. These 20 bug artifacts encompass various cases of regression bugs in real-world. We belive that BugOSS offers a useful basis for empirically investigating regression fuzzing techniques.

Please let us know if you have any question or request for using BugOSS: Jeewoong Kim [email protected] and Shin Hong [email protected]

BugOSS will be presented at the poster session in 2023 IEEE International Conference on Software Testing, Verification, and Validation (ICST). Please use the following bibtex entry when you cite BugOSS.

@INPROCEEDINGS{BugOSS,
     author={Kim, Jeewoong and Hong, Shin},
     booktitle={IEEE International Conference on Software Testing, Verification, and Validation (ICST)}, 
     title={{Poster}: {BugOSS}: A Regression Bug Benchmark for Evaluating Fuzzing Techniques}, 
     year={2023}
}

Each artifact provides a realistic regression bug context to reproduce the reported failure:

• fuzz target: the latest version before the oss-fuzz issue report time which successfully reproduce a failure
• bug-revealing input: an input for a fuzz target, that induces a failure (attached in an OSS-Fuzz issue)
• bug-inducing commit: the program change that newly adds the target bug to the target program
• bug-fixing commit: the program change that repairs the target bug
• bug locations: a subset of the changed lines in the BIC, that are suspected to a failure when the bug-revealing input is given
• fix locations: a subset of the changed lines in the BIC, that are related to the bug-fixing changes at BFC
• bug-specific test oracle: a condition to determine whether a failure is induced by the target bug, or it is induced by other bugs
• other failures by pre-existing bugs
• an initial seed corpus at a bug-inducing commit (seed_corpus.tar)
• experiment results with 2 baseline fuzzers (libFuzzer, AFL++) using the initial seed corpus

1. Clone the BugOSS reprository to your machine

   git clone https://github.com/sdevlab/BugOss.git
   

2. Clone the OSS-Fuzz repository since BugOSS uses docker images of the OSS-Fuzz

   git clone https://github.com/google/oss-fuzz.git
   

3. Copy all files in an artifact directory to a oss-fuzz/projects/project-name/
   all the given files (e.g., fuzz_target.cpp) should be located in the same directory with a Dockerfile, for example:

   cp BugOss/aspell-18462/* oss-fuzz/projects/aspell/ 
   

4. Build the artifact with the given failure-reproducing-information from BugOSS using oss-fuzz/infra/build_specified_commit.py, for example:

   python3 oss-fuzz/infra/build_specified_commit.py 
               --project_name aspell  
               --commit e0646f9b063b23754951f1254f1ecb7af8ca36f3 
               --engine libfuzzer 
               --sanitizer address