Prove it,or break it.

A solver looks for an input the old and new versions disagree on. No specs to write: the old code is the spec.

On a pull request

What it says.

Three outcomes, never rounded into each other.

apply_discount  [smt]  behavior changed

  distinguishing input   price=100, qty=1
  old                    100
  new                    90

Proven: No input exists on which the two versions differ. Not a suite that happened to be green: no such input can be constructed.
Changed: One does exist, and here it is, with what each version returns. A concrete case rather than a warning.
Unknown: The solver could not decide within its budget. Reported as undecided, not dressed up as either of the others.

After the finding

It hands you the test.

The input it found, written out as a test you can commit.

def test_apply_discount_preserves_behavior():
    # the input differential verification found:
    # OLD returned 100, NEW returned 90
    assert apply_discount(100, 1) == 100

The input is worth more than the warning. It becomes a regression test you can commit, so the case that broke the behaviour is the case that stops it breaking again.

Honesty

What proven means here.

Proven means proven about the function as written, under the model the solver works in. A strong claim and a bounded one: it does not cover what happens outside the function, and it does not know what the function was supposed to do.

A narrower promise than most testing makes, and a much harder one. A suite says nobody found a case. This says there is no case to find.

Try Graphify now.

Try for free