deflake.rs

Flaky test detection for Rust

The idea came from my undergraduate dissertation supervisor, Phil McMinn, and is based off the DeFlaker paper by Bell et al.

Features

Uses per test code coverage & diff of changes since previous commit to determine if modifications could have caused a test to fail.

Usage

cargo deflake run tests and classify failures

cargo deflake --help see all configuration options

Requirements

Installation

cargo-binutils, rustfilt and llvm-tools-preview are required at the moment to run the tool cargo install cargo-binutils rustup component add llvm-tools-preview cargo install rustfilt

Examples and testing

cargo deflake

Can run local version against other repos by building and adding /path/to/repository/target/debug to your systems path.

Repository structure

• poc1 - deflake.rs codebase
• myReport - final dissertation report
• project-containers - configuration and scripts used for evaluation
• experiment-helpers - small program used for evaluation
• flaky_example - example project with flaky test used for testing
• repo-searcher - old project used to find Rust projects with flaky test mentions