DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅

Thank you @neerazz, will take a look

I have read the DCO Document and I hereby sign the DCO

Thanks for the review @jmartin-tech. Pushed two fixes:

1. CI failure (test_docs.py::test_docs_probes[adaptiveattacks]) — added the missing docs/source/probes/adaptiveattacks.rst stub and linked it from index_probes.rst. The Linux/Mac/Windows pytest cancellations were all downstream of this single doc-test failure.

2. Full probe duplication concern — AdaptiveAttacksFull is already active = False (opt-in only) on this branch, but the docstring/comment didn't make the design intent obvious. Updated both to call out explicitly that it's inactive by default to avoid prompt duplication when a run selects both AdaptiveAttacks (capped) and AdaptiveAttacksFull (uncapped). Happy to drop Full entirely if you'd prefer the simpler shape — let me know.

Just pushed a85aa90 which applies the canonical pattern more directly per your earlier suggestion:

• AdaptiveAttacks now declares DEFAULT_PARAMS = Probe.DEFAULT_PARAMS | {"follow_prompt_cap": True} and guards the prune call with if self.follow_prompt_cap: self._prune_data(cap=self.soft_probe_prompt_cap) — matching the shape used in encoding.py / latentinjection.py / factor up template soft_probe_prompt_cap logic #1546 / probe: probes should follow soft_probe_prompt_cap #1562.
• AdaptiveAttacksFull now expresses "no cap" via DEFAULT_PARAMS = AdaptiveAttacks.DEFAULT_PARAMS | {"follow_prompt_cap": False} instead of overriding __init__ to bypass AdaptiveAttacks.__init__. Cleaner, configurable, no constructor hack.

active = False on AdaptiveAttacksFull is preserved so the duplication concern stays addressed by default. If you'd still prefer to drop Full entirely rather than ship two probes, happy to do that in a follow-up commit — just let me know.

Local verification (Python 3.12): 8/8 tests/probes/test_probes_adaptiveattacks.py, 4/4 adaptive cases in tests/test_docs.py, 1130/1130 in tests/probes/test_probes.py (excluding the pre-existing AudioAchillesHeel optional-deps failure unrelated to this PR).

Friendly nudge — wanted to make this easy to action on whatever timeline works.

State recap as of today:

• All 16 CI checks ✅ (Linux/Mac/Windows × Py 3.10/3.12/3.13, docs, pip install)
• DCO signed
• @jmartin-tech's Full duplication concern addressed in a85aa90 via the canonical DEFAULT_PARAMS | {"follow_prompt_cap": False} pattern, with active = False preserved so it's opt-in only

Two simple paths from here, whichever you prefer:

1. Ship as-is — AdaptiveAttacks (capped, active) + AdaptiveAttacksFull (uncapped, inactive). Duplication risk is gated by active = False.
2. Drop Full entirely — I'll push a one-commit follow-up removing the second class and keeping only AdaptiveAttacks with the configurable cap. Simpler surface, no opt-in probe to maintain.

Happy to do either. Just let me know which shape you'd rather merge.

Simplified to a single probe per the earlier feedback thread — dropped AdaptiveAttacksFull entirely (commit 3ceba9b).

What changed:

• AdaptiveAttacksFull class removed from garak/probes/adaptiveattacks.py
• Test file updated to remove the Full-specific parametrize entry and the test_adaptiveattacks_full_ignores_cap test
• AdaptiveAttacks (capped, active=True) is the only probe now — cleaner surface, no inactive opt-in class to maintain

Uncapped runs still possible via --config follow_prompt_cap=false (the DEFAULT_PARAMS key is still there).

This matches option 2 from my earlier comment. All 16 CI checks were green on the previous commit; no functional change to AdaptiveAttacks itself, so CI should stay green.

Fixed DCO sign-off. Amended commit `3ceba9b` → `c2f2ae3` to add the missing `Signed-off-by` line:

```
Signed-off-by: Neeraj Kumar Singh [email protected]
```

Force-pushed to `probe-adaptive-attacks-583`. DCO check should now pass. @jmartin-tech @leondz — ready for your review when you have a chance.

Thanks for the review, @jmartin-tech!

Alias tweak (commit 91b70748): removed aliases = [...]. You're right — aliases are for renamed probes and a new probe should have none. Applied and force-pushed.

Recent target with measurable success rate:

The technique this probe implements (adaptive template + cached suffix) is directly from Andriushchenko, Croce, and Flammarion (ICML 2024, arxiv:2404.02151). The paper reports concrete attack success rates against several safety-aligned models that remain in active deployment today: GPT-3.5-turbo, GPT-4, Gemini Pro, and Llama-2-7B/13B-chat. Those are the primary documented targets with published success rates.

For a more recent reference: within the past year, Meta Llama 3 variants (released April 2024) and derivative fine-tuned models have been widely studied as jailbreak targets and are commonly used in garak evaluation runs against this class of prompt-injection attack. Standard RLHF safety training without specific defenses against template-based adaptive prompting continues to show non-trivial attack success rates with this technique.

If you'd prefer a specific success rate number on a 2025-era model, I can run the probe against an open-source model (e.g., Llama-3.1-8B-Instruct) and report the output — but I didn't want to delay the PR for that if the paper's citations and the broader Llama 3 evaluation context are sufficient for your purposes. Let me know.

Hi @jmartin-tech — wanted to follow up since my Jun 9 response may have gotten buried. All items from your review are addressed:

• ✅ aliases removed (commit 91b70748)
• ✅ AdaptiveAttacksFull dropped; single capped probe, fully configurable cap via --config follow_prompt_cap=false (commit 90eb9f99)
• ✅ DCO sign-off fixed
• ✅ CI is fully clean (all checks passing)

On your question about a recent target with measurable success rate — I replied with references to the ICML 2024 paper targets (GPT-3.5/4, Gemini Pro, Llama-2) and Llama 3 variants. If you'd find it more useful to have a direct number from running the probe against Llama-3.1-8B-Instruct, I'm happy to do that before you close the review — just say the word.

Whenever you get a chance to re-review, I'd really appreciate it. Thanks again for the detailed feedback that made this a cleaner, simpler probe.

Thanks @jmartin-tech — those revisions sound good to me. Please feel free to make the adjustments before landing.

Minor adjustment pushed, maintainer discussion landed on naming convention as adaptive_attacks.AdaptiveAttacks. This will land once CI validation completes.