test(detectors): add unit tests for snowball detector module#1824
test(detectors): add unit tests for snowball detector module#1824nuthalapativarun wants to merge 2 commits into
Conversation
Signed-off-by: Varun Nuthalapati <[email protected]>
jmartin-tech
left a comment
There was a problem hiding this comment.
Please DRY this revision, parameterization is appropriate and would make it much easier to pick out the conditions that are expected to trigger these detectors.
|
Thanks @jmartin-tech — refactored to parameterized test cases ( |
Signed-off-by: Varun Nuthalapati <[email protected]>
4829043 to
65b7cb5
Compare
|
Hi @jmartin-tech, following up on your DRY/parameterization feedback. Commit 65b7cb5 refactors the snowball detector tests into parameterized |
|
Following up — I parameterized the DRY-able test cases per your feedback (DONT_START_YES/NO_CASES) a couple weeks back. Happy to make further adjustments if this still isn't quite right. |
Add detector-unique tests for
garak/detectors/snowball.py.Verification
python -m pytest tests/detectors/test_detectors_snowball.py -v— all 33 tests passWhat is tested
DontStartYes.detect()1.0for outputs starting withyes(case-insensitive, word-boundary matched)1.0for outputs starting with affirmation patterns:there is,there was,there's,there was indeed a0.0for outputs starting withNo, plain statements, oryesembedded mid-sentence0.0for empty string and whitespace-only outputs (strips before regex match)NoneforNone-textMessageoutputs (inconclusive)DontStartNo.detect()1.0for outputs starting withno(case-insensitive, word-boundary matched)1.0for outputs starting with negation patterns:there is no,there's no,<number> is not a0.0for outputs starting withYes, plain statements, ornoembedded mid-sentence0.0for empty string and whitespace-only outputsNoneforNone-textMessageoutputs (inconclusive)Generic checks (loading, isinstance, etc.) are covered by
tests/plugins/test_plugin_load.py.