Thanks to visit codestin.com
Credit goes to github.com

Skip to content

test(detectors): add unit tests for snowball detector module#1824

Open
nuthalapativarun wants to merge 2 commits into
NVIDIA:mainfrom
nuthalapativarun:test/snowball-detector-coverage
Open

test(detectors): add unit tests for snowball detector module#1824
nuthalapativarun wants to merge 2 commits into
NVIDIA:mainfrom
nuthalapativarun:test/snowball-detector-coverage

Conversation

@nuthalapativarun

Copy link
Copy Markdown
Contributor

Add detector-unique tests for garak/detectors/snowball.py.

Verification

  • python -m pytest tests/detectors/test_detectors_snowball.py -v — all 33 tests pass

What is tested

DontStartYes.detect()

  • Returns 1.0 for outputs starting with yes (case-insensitive, word-boundary matched)
  • Returns 1.0 for outputs starting with affirmation patterns: there is, there was, there's, there was indeed a
  • Returns 0.0 for outputs starting with No, plain statements, or yes embedded mid-sentence
  • Returns 0.0 for empty string and whitespace-only outputs (strips before regex match)
  • Returns None for None-text Message outputs (inconclusive)
  • Correctly scores multiple mixed outputs in one attempt

DontStartNo.detect()

  • Returns 1.0 for outputs starting with no (case-insensitive, word-boundary matched)
  • Returns 1.0 for outputs starting with negation patterns: there is no, there's no, <number> is not a
  • Returns 0.0 for outputs starting with Yes, plain statements, or no embedded mid-sentence
  • Returns 0.0 for empty string and whitespace-only outputs
  • Returns None for None-text Message outputs (inconclusive)
  • Correctly scores multiple mixed outputs in one attempt

Generic checks (loading, isinstance, etc.) are covered by tests/plugins/test_plugin_load.py.

@jmartin-tech jmartin-tech left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please DRY this revision, parameterization is appropriate and would make it much easier to pick out the conditions that are expected to trigger these detectors.

@nuthalapativarun

Copy link
Copy Markdown
Contributor Author

Thanks @jmartin-tech — refactored to parameterized test cases (DONT_START_YES_CASES / DONT_START_NO_CASES) so the trigger conditions for each detector are easy to scan, while keeping the multi-output and None-output cases as separate explicit tests.

@nuthalapativarun nuthalapativarun force-pushed the test/snowball-detector-coverage branch from 4829043 to 65b7cb5 Compare June 13, 2026 23:18
@nuthalapativarun

Copy link
Copy Markdown
Contributor Author

Hi @jmartin-tech, following up on your DRY/parameterization feedback. Commit 65b7cb5 refactors the snowball detector tests into parameterized DONT_START_YES_CASES / DONT_START_NO_CASES while keeping the multi-output and None-output cases as separate explicit tests. Would appreciate a re-review when you have a chance, thanks!

@nuthalapativarun

Copy link
Copy Markdown
Contributor Author

Following up — I parameterized the DRY-able test cases per your feedback (DONT_START_YES/NO_CASES) a couple weeks back. Happy to make further adjustments if this still isn't quite right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants