Find Highs Before External
Auditors
Using Invariant Fuzz Testing & Formal Verification
dacian.me | x.com/DevDacian | cyfrin.io | Fuzz Fest - Dec 2024
Observation:Many Highs found during private
audits by external auditors could have been
found by the protocol developers with
invariant fuzzing prior to engaging external
auditors. This doesn’t require “attacker
mindset” or other auditor-specific skills,
But does require learning to think in
invariants.Economics:Developers already
have full context of protocol and important
properties. Hence it is more cost-effective for
protocol developers to implement fuzzing
suites prior to engaging more expensive
external audit firms.
Benefits For Auditors:Learn
a new thinking method to
view protocol in terms of
invariants
=> Find new bugs you
would have missed before
Everyone can learn thisYOU
can learn this
Simple invariants can find
complicated bugs
Thinking In Invariants
Smart Contract Lifecycle, Black/White-Box
Invariant = useful protocol-specific property that must remain true, implemented as:-
Echidna/Medusa : function that returns true if property holds, false otherwise- Foundry
: function that contains one or more assertions using assertEq, assertTrue etc
Smart Contract Lifecycle - construction/initialization, regular functioning, optional end
state
Black Box - property we can glean from the protocol’s design or documentation,
without needing to know about the internals of how the protocol actually implements
functionality(note - related black-box testing)
White Box - property we implement based upon internal knowledge of how the smart
contract works (by reading the code, examining storage & how functions manipulate
it etc)
Thinking In Invariants
Sanity Check / System Property / Valid State
Basic invariant = obvious sanity check- Eg: sum of initial allocated points must equal total points
(initialization, white-box)
Advanced 1) relationship between different storage locations- Eg: the sum of all values in a mapping X
must equal to Y stored elsewhere in storage (regular functioning, white-box)- Eg: all addresses present
in EnumerableSet X must also exist as keys in mapping Y (regular functioning, white-box)
Advanced 2) monetary value stored in contract, solvency- Eg: once token distribution is complete, the
contract should have 0 balance (end state, black-box)- Eg: contract should always have enough tokens
to cover liabilities (regular functioning, white-box)
Advanced 3) logical invariant- Eg: account with active borrow can’t exit market (regular functioning,
black-box)- Eg: protocol should never enter a state where borrower can’t repay a loan but can be
liquidated (regular, black box)
Advanced 4) denial of service- DoS invariant fuzz tests detect subtle bugs due to unexpected reverts
from access array out of bounds, under overflow etc (regular functioning, white-box as we need to
know valid reverts)
Thinking In Invariants
Handlers vs No Handlers
Fuzzers can use handlers which wrap underlying functions from target contract,
satisfying preconditions. Alternatively fuzzers can have no handlers which lets
fuzzer explore freely
No handlers- if search space is too large, fuzzer may not have enough time to find
an invariant-breaking sequence- can have many wasted runs due to trivial reverts
Handlers- eliminates wasted runs due to trivial reverts by satisfying underlying
function preconditions- restricts search space so can miss some attack paths
Invariant Examples* Code ->
Invariant* Specification -
>Invariant* First example = CTF
* All the rest = Simplified
versions of real findings from
my private audits
@CyfrinAudits
Code -> Invariant Code:
user should always be able to take a flash loanas long as there is enough eth in the pool (regular
functioning)
BlackBox invariant - no knowledge ofimplementation
Whitebox invariant - using implementationdetails of specific check code is doing
Code -> Invariant
user should always be able to take a
flash loanas long as there is enough
eth in the
BlackBox pool (regular
invariant functioning)
- no knowledge ofimplementation
Find the same bug using Certora
Formal Verification- no setup,
handlers or other fuzzer
Whitebox invariant
“infrastructure” - using -implementationdetails
required very of specific check code is doing
concise!- use a powerful “parametric
rule” to describe a “scenario” to the
Prover- use `require` statements to
define relationships between entities
to constrain valid initial state
Spec -> Invariant Code:
either proposal still active and contract has ethor proposal has been decided and
eth has beendistributed or refunded (regular function + end of life)
BlackBox invariant as it does not make use of implementation logic about how the
contract handles distribution, just checks balances and active state
Spec -> Invariant Certora
either proposal still active and contract has ethor proposal has been decided and
eth has beendistributed or refunded (regular function + end of life)
Spec -> Invariant
Specification:- NFT has voting power. Users have to deposit required ETH Collateral
against NFT otherwise power reduces over time- NFTs can be created by owner
until "power calculation start timestamp". Power calculation only starts from this
timestamp- NFTs start with "max power"
Invariant - When NFT powercalculation starts, total maxpower must equal sum of
allcreated NFTs (initialization, blackbox)
Spec -> Invariant
Vesting contract allocates points to users which can be used to redeem tokens
“Cheap” invariants can be implemented directly into the code; verify all points
allocated at initialization
“Expensive” invariant checked in fuzztest - sum of all user points always equalto
total points (regular function, blackbox)
Spec -> Invariant Certora
Vesting contract allocates points to users which can be used to redeem tokens
“Cheap” invariants can be implemented directly into the code; verify all points
allocated at initialization
“Expensive” invariant checked in fuzztest - sum of all user points always equalto
total points (regular function, blackbox)
Specification:- Token Sale allows a DAO to sell its governance token ("sellToken")
for another token (“buyToken")- Token sale only for allowed users and each user
can only buy up to hard limit- for simplicity exchange rate 1:1
Invariant - number of tokens bought matchesnumber of tokens sold (regular
function, blackbox)
Invariant - each user can only buy up to the hardlimit of tokens (regular function,
blackbox)
Chimera - Multi Fuzzer Invariants
Use Echidna, Medusa & Foundry with same code
Set of contracts to inherit from to create re-usable fuzzing code:- contract Setup is
Chimera::BaseSetup (implement setup & ghost variables)- contract Properties is
Chimera::Asserts, Setup (define invariants)- contract TargetFunctions is
Chimera::BaseTargetFunctions, Properties (handlers)- contract CryticTester is
Chimera::CryticAsserts, TargetFunctions (Echidna/Medusa
front-end)- contract
CryticToFoundry is Chimera::FoundryAsserts, TargetFunctions, ForgeStd::Test
(Foundry front-end)
You define 5 contracts which each inherit from Chimera’s parent contracts and can
then use the same code with Echidna, Medusa & Foundry Fuzzers
See https://dacian.me/writing-multi-fuzzer-invariant-tests-using-chimera
Exercise #1
Vesting Extended
A `VestingExt` contract allocates points to users which can be redeemed for
tokens once the vesting period has expired
Users can also `preclaim` a smaller amount of their total token allocation prior to
the expiry of their vesting period
One invariant already exists to verify users can’t artificially increase their points
Your mission is to write:- an additional handler for the `preclaim` function- an
additional invariant to verify `preclaim` correctness
Exercise #1 Certora
Vesting Extended
Exercise #2
Operator Registry
An `OperatorRegistry` contract allows users to register as operators and receive
an `operatorId`
Users can also update their registered address
Your mission is to write an invariant that verifies the integrity between multiple
inter-related data structures
Exercise #2 Certora
Operator Registry
Exercise #3
Liquidate Denial of Service
A `LiquidateDos` contract allows usersto enter different markets and have one
open position in each market
Users can also be liquidated
Your mission is to write an invariant whichverifies that the liquidate function won’t
revert with any unexpected errors
Exercise #4
Stability Pool
A `StabilityPool` contract where users can deposit `debtToken` and receive a share of
`collateralToken` rewards from liquidations
Contains quite complicated logic to calculate the rewards users should receive
Your mission is to write an invariant which verifies that the `StabilityPool` always
remains solvent; that it always has enough `collateralToken` to pay out rewards to
depositors
This is a black-box invariant as it requires no knowledge of the complex reward
calculations
This exercise shows one great benefit of invariant fuzzing; that even without
understanding the complex reward calculation code, the fuzzer can find a critical
exploit for you if only you provide it with a simple black-box invariant
Exercise #4
Stability Pool
Exercise #5
Collateral Priority
A `Priority` contract used in multi-collateral lending protocols
Defines a priority order for collaterals such that when a liquidation occurs, the
riskiest collaterals will be liquidated first such that the borrower’s remaining
collateral basket is more stable post-liquidation
Collateral can be added or removed
Your mission is to write an invariant to verify that the priority order is always
maintained
Exercise #5
Collateral Priority
Exercise #5 Certora
Collateral Priority
Bonus - Rarely False
Bonus -
Stateless Fuzz Vault
First Deposit
“Make sure your fuzz
tests are good first”
Patrick Collins | x.com/PatrickAlphaC | cyfrin.io
