v1.5.0 — multi-step cockpit orchestration + BSS_REPL_LLM_AUTONOMY#43
Merged
Conversation
…iring New orchestrator/bss_orchestrator/autonomy.py module mirroring loyalty-cli's read_autonomy_mode + AutonomyMisconfigured. Single env var BSS_REPL_LLM_AUTONOMY, two valid values (granular default / batched), fail-closed validation at process boot. Case + whitespace normalised before validation; empty/unset returns the default; anything else raises AutonomyMisconfigured with a message that names the env var, the bad value, and the valid set. Wired into both cockpit boot paths: - portals/csr/bss_csr/main.py lifespan reads + caches on app.state and logs the resolved mode at "cockpit.starting" - cli/bss_cli/repl.py _bootstrap_store_and_config() reads + stashes in a module-level cache so _drive_turn doesn't have to re-parse env per turn (and per-session override in v1.5.1 has an obvious touch point) Both paths let AutonomyMisconfigured propagate — same fail-closed shape as the v0.9 named-token sentinel rejection. No silent default-on-typo. 20 new unit tests in orchestrator/tests/test_autonomy.py cover: unset + empty + whitespace-only default to "granular"; both valid modes load; case/whitespace normalisation (GRANULAR / Batched / " granular "); 10 unknown-value rejection cases including typos (granuler, batch); error message must name env var + bad value + valid set; canonical-set guard. Behaviour-preserving: nothing reads the cached mode yet. Phase B wires it into the destructive-tool gating in safety.py + session.py. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
safety.py — wrap_destructive grows two new kwargs:
- autonomy_mode: "granular" or "batched" (default "batched" to preserve
pre-v1.5 behaviour for the dozens of test/scenario callers that don't
pass it; production cockpit paths in v1.5+ pass it explicitly)
- loop_state: per-graph mutable LoopState dict, created fresh by
build_tools per build_graph invocation and shared across every
destructive wrapper in that graph
In granular mode + allow_destructive=True: the FIRST destructive call
fires; subsequent destructive calls in the same graph block again with
the same structured DESTRUCTIVE_OPERATION_BLOCKED response, forcing the
LLM to re-propose so the operator can /confirm each one. Per-step
operator control even after the first authorisation.
In batched mode + allow_destructive=True: every destructive in the
graph fires — historical pre-v1.5 behaviour, preserved for callers
that want one /confirm to authorise the loop.
graph.py — build_tools + build_graph + _as_structured_tool plumb
autonomy_mode + a freshly-created LoopState through to every wrapped
destructive tool in the graph. Each astream_once invocation builds its
own graph, so LoopState naturally resets between turns.
session.py — astream_once + ask_once grow an autonomy_mode kwarg
(default "batched" for backward compat) and surface it on the OTel
span as bss.ask.autonomy_mode for trace-level observability.
portals/csr cockpit route + cli REPL — read the cached autonomy mode
(app.state.autonomy_mode / module-level _AUTONOMY_MODE respectively)
and pass it into astream_once. Both paths set the cached value at
boot from read_autonomy_mode().
6 new safety tests + the existing 4 still pass:
- granular blocks second destructive even when allow_destructive=True
- batched runs every destructive once authorised
- granular doesn't relax the disallowed case
- non-destructive calls don't consume the granular budget
- fresh LoopState resets between graph builds (= between turns)
- default autonomy is "batched" for the primitive (back-compat guard)
make test: orchestrator 403→409, cli 127, bss-cockpit 66 — all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Catches the Gemma thrash pattern where the model fires the same broken call over and over instead of adapting. After 3 consecutive failure- shaped tool results, the stream terminates with a structured AgentEventError so the cockpit can render a "couldn't recover" panel instead of letting the loop spin forever. Counter resets on any success. "Failure" = ToolMessage.status="error" (LangGraph exception path) OR the result body contains one of three structured-error markers: DESTRUCTIVE_OPERATION_BLOCKED, POLICY_VIOLATION, CLIENT_ERROR. The marker check is conservative — false negatives just delay or skip the bail; false positives could end a healthy investigation loop early, so the marker must be a literal "key: value" substring match (not a loose "error" word check). MAX_CONSECUTIVE_TOOL_FAILURES = 3 — direct lift from loyalty-cli. 8 new unit tests cover the classifier surface: - status="error" trips regardless of body - each structured-error marker trips - a customer email containing "error" does NOT trip (false-positive guard) - empty/normal results don't trip - threshold constant guard so the test surfaces if someone changes it End-to-end bail behaviour will be exercised in Phase E's e2e suite — testing the loop integration here would require either a real graph (slow + non-deterministic) or a heavy mock. The classifier is the small surface most likely to drift. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
New packages/bss-cockpit/bss_cockpit/chrome_filter.py with:
- is_cockpit_chrome(content): True for cockpit-emitted assistant
strings (route error fallback "Sorry — something went wrong...",
Gemma empty-final recovery "(The model called X but did not
synthesise...)", total-empty fallback "(no reply)", citation-
guard fallback "I don't have a citation for that..."). Empty /
whitespace-only content also classified as chrome (the cockpit
never persists empty real replies — they get replaced before
persistence).
- strip_fake_propose(text): runtime backstop for LLMs that mimic
the cockpit's "⚠ PROPOSE: ..." banner shape as plain text
(the anti-mimicry prompt rule is the first line of defence).
Wired into Conversation.transcript_text() so the cockpit's own
emit chrome is filtered out before astream_once(transcript=...) sees
it. Without the filter, the LLM reads its own past placeholder
bubbles as prior reasoning and three failure modes follow: mimicry
(emits new turns in the same chrome shape), state confusion (sees
"did not synthesise" and re-tries the same broken call), and
citation thrash (learns the fallback as the default).
Conservative prefix-match design — false negatives just leave benign
noise; false positives would strip real LLM output. Pattern lifted
from loyalty-cli's _is_cockpit_chrome / _strip_fake_propose; bss-cli
chrome strings differ but the design is identical.
22 new chrome-filter unit tests:
- every known cockpit prefix recognised (parametrised)
- empty/whitespace classified as chrome
- 9 legitimate LLM replies NOT misclassified (false-positive guard)
- fake-PROPOSE stripper handles step variants, case-insensitive,
Unicode warning glyph, leaves real prose alone
- inventory lock: _ASSISTANT_CHROME_PREFIXES contents pinned so a
new cockpit fallback added without a matching prefix surfaces
in code review
make test: bss-cockpit 66→87 — all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…E FLOW
Two edits to _COCKPIT_INVARIANTS in packages/bss-cockpit/bss_cockpit/
prompts.py:
1. Soften the v0.19 "after renderer-backed call → one short sentence,
STOP" rule. The TEXT reply that TERMINATES a compound action is
still bound by the one-sentence + no-re-render rule. But the agent
MAY now emit ANOTHER TOOL CALL instead of a text reply when the
prompt clearly requires more steps — the one-sentence rule kicks in
only on the terminating reply. Anti-duplication contract unchanged.
2. Add the ITERATIVE FLOW block as a new (v1.5+) rule, with three
bss-shaped worked examples:
- "show me CUST-001 and then list their subscriptions" → reads
chain in one turn, terminate with a brief one-sentence summary
- "investigate CASE-042" → case.get → customer.get →
subscription.list_for_customer → ticket.list (any order), end
with situational summary + proposed next operator action; do NOT
propose destructive tools unless the operator's prompt explicitly
asks for one (key doctrine — agent doesn't escalate on its own)
- "register CUST-XYZ then issue PLAN_M" → compound write, gets
re-gated under granular autonomy and runs inline under batched
Plus explicit guidance on tool-error adaptation (the 3-strike bail
is the safety rail; ask for help on attempt #2 rather than letting
it fire) and a reminder to terminate with a brief text message.
Doctrine guard in orchestrator/tests/test_iterative_flow_scope.py:
- customer_chat_prompt.py MUST NOT contain "ITERATIVE FLOW"
(compound actions are an operator capability, not a customer-
chat one — the v0.12 chat caps + ownership trip-wire were never
stress-tested against agent-driven write chains)
- bss_cockpit/prompts.py MUST contain "ITERATIVE FLOW" (guard
against accidental removal regressing v1.5's main unlock)
- The block must be tagged with the v1.5 marker for archaeology
make test: orchestrator 417→420 — all green.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…tonomy
CLAUDE.md anti-patterns gain a new Cockpit (v1.5 — multi-step autonomy)
block with four entries: autonomy module ownership (don't read
BSS_REPL_LLM_AUTONOMY outside bss_orchestrator.autonomy); ITERATIVE
FLOW scope (must not leak into customer_chat_prompt.py, guarded by
orchestrator/tests/test_iterative_flow_scope.py); MAX_CONSECUTIVE_TOOL_
FAILURES contract (3, lifted from loyalty-cli); _ASSISTANT_CHROME_
PREFIXES inventory lock (new cockpit fallback bubble = new prefix entry).
ARCHITECTURE.md § Operator cockpit gains a v1.5 paragraph covering the
two autonomy modes + their /confirm semantics, the read_autonomy_mode
fail-closed boot contract, the 3-strike loop bail, and the cockpit-
chrome filter. Notes that DESTRUCTIVE_TOOLS is unchanged — autonomy
controls how many /confirms a compound action needs, not which tools
require one.
HANDBOOK.md:
- env var table gains BSS_REPL_LLM_AUTONOMY row pointing at §8.20
- new §8.20 "Multi-step cockpit actions (v1.5)" with granular vs
batched worked examples, abort semantics, recovery + 3-strike
bail rule, chrome-filter contract, and an opinion on when to flip
to batched (mirrors loyalty-cli HANDBOOK §8.16 shape)
README.md gains a "Multi-step cockpit actions (BSS_REPL_LLM_AUTONOMY,
v1.5)" subsection between the e2e block and the Documentation map.
.env.example BSS_REPL_LLM_AUTONOMY=granular entry with explanation of
both modes and the fail-closed boot contract.
DECISIONS.md 2026-05-26 v1.5.0 entry covers the five-piece scope
(autonomy env, autonomy-aware gating, ITERATIVE FLOW prompt + softened
v0.19 rule, 3-strike bail, chrome filter), the three rejected
alternatives (plan-emit-then-confirm-once, composite read tools,
per-tool autonomy annotations), and the consequences (compound actions
work in one prompt under batched, doctrine/handbook updates, deferred
v1.5.1/v1.5.2 follow-ups).
make test still green: orchestrator 420, cli 127, bss-cockpit 87.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Three new Playwright specs in packages/bss-e2e/tests/test_v15_multi_step.py
exercise the v1.5 multi-step unlock end-to-end through the cockpit
browser veneer:
1. test_v15_compound_action_granular — operator types a compound
prompt, turn 1 stages pending_destructive(A), /confirm POSTs,
turn 2 executes A and the granular wrapper re-blocks B,
cockpit stages pending_destructive(B). The "second pending row
appears after /confirm" is the v1.5 signal.
2. test_v15_compound_action_batched — same flow, but under
BSS_REPL_LLM_AUTONOMY=batched the loop executes BOTH destructives
autonomously and zero pending rows remain. Skipped by default
(granular is the v1.5 doctrine default); run via the new
`make e2e-batched` target.
3. test_v15_case_investigation — chained reads in ONE turn
(case.list → customer.list → catalog.list_active_offerings →
one-sentence summary). Tests that v0.19's "Done." rule is
softened (the agent chains the three reads in one turn) AND
that the anti-duplication contract is intact (no markdown
pipe-table re-render of the data the cockpit already showed).
Required fixture entries in packages/bss-e2e/fixtures/cockpit_e2e.json:
- v15-compound-action-turn1-propose-first-destructive (matches
"plan compound")
- v15-compound-action-turn2-after-confirm-tries-two-destructives
(matches "operator typed /confirm" — the synthetic prompt the
cockpit injects on /confirm-resumed turns)
- v15-case-investigation-chained-reads (matches "investigate the
demo state")
All three use provisioning.set_fault_injection / read-only tools
with safe args so the suite can run repeatedly without dependent
fixtures.
Route + REPL fix — pending_destructive STAGING now derives from
COMPLETED-with-BLOCKED, not STARTED. The pre-v1.5 path set
last_proposal on STARTED for every destructive when
allow_destructive=False; that heuristic missed the v1.5 granular
re-gate (which fires on a /confirm-resumed turn where
allow_destructive=True yet the wrapper still BLOCKS the second
destructive). The new path observes the tool RESULT (BLOCKED
sub-string in event.result_full), captures the matching args via
call_id round-trip on captured_tool_calls, and unconditionally
stages whenever a destructive's result is BLOCKED. Covers both
paths cleanly:
• pre-v1.5 (allow=False blocks at the gate) → BLOCKED → stage
• v1.5 granular (second destructive in /confirm-resumed loop
re-gates with allow=True) → BLOCKED → stage
• v1.5 batched (both destructives execute) → no BLOCKED → no stage
The staging guard at the bottom of each handler also drops the
`not allow_destructive_this_turn` condition that would have
silently swallowed case 2.
docker-compose.e2e.yml propagates BSS_REPL_LLM_AUTONOMY into
portal-csr with a granular default (Compose env-var interpolation).
Makefile gains a thin `e2e-batched` target that pre-exports the
env before invoking the existing `e2e` recipe.
make test still green: orchestrator 420, cli 127, bss-cockpit 87.
E2E specs themselves require docker compose + Playwright — run
`make e2e` (or `make e2e-batched` for the batched spec) to exercise.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…ched green
Two real bugs surfaced when running the v1.5 specs against a live stack:
1. Fixture args for provisioning.set_fault_injection were wrong against
the actual tool signature. LangChain pydantic-validates structured
tool args BEFORE the destructive wrapper runs, so the gate never
even fired — the tool produced "Error invoking tool ... task_type:
Input should be 'HLR_PROVISION' | ..." validation errors and the
COMPLETED-with-BLOCKED detector (correctly) didn't trigger. Fixed
by using the real enum values:
- task_type: HLR_PROVISION / ESIM_PROFILE_PREPARE (not the made-up
create_msisdn / create_esim_profile)
- fault_type: fail_first_attempt (not fault_kind: fail)
- enabled: false (was missing — required field)
This is also why the pre-v1.5 test_cockpit_propose_then_confirm
spec "passed" before my route handler change: it only asserted on
the STARTED-side last_proposal capture, which fired regardless of
whether the wrapper was reached. Now that staging derives from
COMPLETED-BLOCKED, the fixture args have to actually validate so
the wrapper actually runs and BLOCKs.
2. The granular + batched specs used POST /cockpit/<id>/confirm to
simulate /confirm, but that endpoint is a no-op marker (per its
own docstring) — it doesn't add a synthetic user message or
trigger a new turn. The actual /confirm-resumed turn fires only
when the operator TYPES /confirm via the compose form (which the
cockpit's slash-command interceptor at routes/cockpit.py:755
catches and rewrites). Switched both specs to use _send_message
(page, "/confirm") + _wait_for_turn_done — same path a real
operator takes in the cockpit.
Live verification:
- make e2e (granular default): 12 passed, 1 skipped (batched gated
on env), 0 failed in 37.76s. All 10 v1.4.1 specs + the new
granular + case-investigation specs green.
- make e2e-batched: 12 passed, 1 skipped (granular gated when
env=batched), 0 failed in 38.61s. Batched spec exercised the
autonomous-second-destructive path successfully.
Both autonomy modes now have real end-to-end coverage on the
cockpit browser veneer.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
… warning
Live REPL session caught Gemma narrating the proposal as prose instead of
emitting a real tool_call:
> terminate subscription SUB-0005
╭─ bss ai ───────────────────────────────────────────────────────╮
│ I propose to terminate subscription SUB-0005. │
│ subscription.terminate(subscription_id="SUB-0005") │
│ │
│ Please type /confirm to proceed. │
╰────────────────────────────────────────────────────────────────╯
> /confirm
> <-- silence; no pending row was ever staged
The cockpit had three holes, all fixed here:
1. **Prompt rule.** Loyalty-cli's anti-mimicry doctrine ported into
bss_cockpit/prompts._COCKPIT_INVARIANTS as a (v1.5+) CALL TOOLS —
DO NOT NARRATE THEM block. Names the failure mode explicitly with
bad/good examples, drawn from the actual Gemma output above.
Bad: "I propose ... subscription.terminate(...)", "PROPOSE: ...",
"Proposed action: ... Type /confirm to proceed." Good: emit the
structured tool_call (cockpit owns the banner formatting), OR ask
cleanly in plain prose when more info is needed.
2. **Runtime strip — broader.** chrome_filter.strip_fake_propose was
pre-this-commit only catching banner-shape mimicry ("PROPOSE:" /
"⚠ PROPOSE:" line prefixes). Now it also catches narrated-call
shape: any \b<lower>.<lower>(...)\b pattern in prose, plus the
"Please type /confirm" boilerplate that wraps it. Conservative
trade-off — a legitimate "subscription.state field" prose mention
survives (no parens), but "subscription.terminate(SUB-0001)" in
prose gets stripped. False-negative cost (chrome leaks through) >
false-positive cost (one fragment of legitimate prose lost).
Wired into the live-emit path in BOTH portals/csr cockpit route
AND cli/bss_cli/repl turn driver; previously only used on history
rehydration. When strip leaves the bubble empty AND no real
tool_call fired this turn → surface an explicit "stalled" warning
instead of an empty bubble that hides the doctrine failure.
3. **/confirm slash handler peek-and-warn.** cli/bss_cli/repl
/confirm slash handler used to be a no-op marker (returned a
"_confirmed_marker" string that was never consumed). Now peeks
cockpit.pending_destructive via the new
Conversation.peek_pending_destructive (read without consuming —
_drive_turn still owns the actual pop on the next message). If
a row exists, reports what's staged + how to drive it. If NO row
exists, warns loudly with the most common cause (mimicry-narrated
propose). Catches the operator-typed-/confirm-into-the-void
silent-bug UX.
5 new chrome_filter unit tests pin the v1.5 narrated-call detector
against verbatim Gemma output + edge cases (multiple tools in one
prose burst, dotted prose without parens left alone, prose around
the call survives the strip).
Verified live:
- make test: orchestrator 420, cli 127, bss-cockpit 87 → 91, all
other suites unchanged. All green.
- make e2e (with rebuilt portal-csr): 12 passed, 1 skipped, 0
failed in 39.78s. Existing specs unaffected by the broader strip.
REPL test: the operator's exact session shape now produces (a) the
strip removes the narrated call from the bubble, (b) the operator's
follow-up /confirm reports "Nothing pending /confirm" with the
mimicry-explanation hint.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Second pass on the v1.5 anti-mimicry defences after live REPL retest
still showed a half-finished propose bubble:
> terminate subscription SUB-0005
╭─ bss ai ─────────────────────────────────────────────╮
│ I propose to terminate subscription SUB-0005. `` │
╰──────────────────────────────────────────────────────╯
> <-- silence; operator confused; /confirm into the void
Three issues this commit addresses:
1. **Strip didn't go far enough.** The narrated-call regex caught
`subscription.terminate(subscription_id="SUB-0005")` but the LLM
had wrapped it in backticks; only the call body got stripped, the
surrounding backticks survived as ``. The leading
"I propose to terminate subscription SUB-0005." narration also
survived. Half-stripped propose still misleads.
Fixes:
- Broaden _NARRATED_CALL_RE to consume optional surrounding
backticks (1-3 ticks, matches inline `call(...)` and triple-
backtick code fences).
- New _NARRATION_LEAD_RE strips "I [propose|will|would like|
intend|am going] to [terminate|cancel|close|remove|refund|
revoke|propose|call|invoke] ..." sentences when adjacent to
a stripped call (only runs if a call was actually removed —
keeps false-positive risk low).
- New _EMPTY_BACKTICK_RE sweeps any leftover empty `` / ``` pairs.
2. **Warning only fired on empty bubbles.** The previous condition
was `mimicry_stripped AND not final_text.strip() AND not
captured_tool_calls`. The user's bubble was non-empty after the
first-pass strip, so the warning never fired and the misleading
text reached the operator. New condition:
no tool_call this turn AND (mimicry_stripped OR mentions /confirm)
→ fire the warning AND replace the bubble with the canonical
explanation. Independent of whether prose remains — a half-
finished propose with /confirm intent is the failure shape we're
catching. Replacement (not append) so the persisted transcript
doesn't feed Gemma its own half-finished narration on the next
turn's rehydration.
3. **Prompt rule was too soft.** Rewrote the destructive contract +
anti-mimicry block in _COCKPIT_INVARIANTS:
- Destructive contract opener now leads with "MUST be EMITTED AS
STRUCTURED tool_calls — never written as prose." (was: "must be
PROPOSED first with a one-line summary and the exact tool name +
arguments", which Gemma read as "write a one-line text summary").
- Anti-mimicry block opens with "**This is the most common doctrine
failure on small models.**" and gives a verbatim BAD example
(the exact "I propose ... subscription.terminate(...) Please
type /confirm" shape Gemma produced) followed by the explicit
failure mode ("the cockpit reports 'no pending action — the
model narrated...'").
4 new chrome_filter tests pin the v1.5 follow-up shapes:
- backtick-wrapped call from live REPL → strip removes call + ticks
+ "I propose" narration; bubble ends empty (caller's warning fires)
- triple-backtick code fence variant
- narration regex requires destructive canon AND "to" (false-positive
guard — "I'm going to think" survives intact)
- narration only fires when a call was stripped this turn
Verified:
- make test: orchestrator 420, cli 127, bss-cockpit 91→95, all
other suites unchanged. All green.
- make e2e (rebuilt portal-csr): 12 passed, 1 skipped, 0 failed
in 39.55s. Existing specs unaffected by the broader strip + new
warning trigger.
REPL retest expectation: same input "terminate subscription SUB-0005"
now produces either (a) a real tool_call and a Pending /confirm
yellow line OR (b) the explicit "no pending action — model narrated
proposal in prose" warning + canonical replacement bubble. No more
silent stalls.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…r BLOCK
Live REPL retest revealed two more UX bugs:
1. **REPL /confirm was informational-only.** My previous fix made
/confirm "peek and report" but never DROVE a turn — so after
/confirm the operator had to type a second prompt to actually
re-evoke the destructive. That broke the workflow because Gemma
rarely re-emits a destructive without explicit prompting; the
pending row sat unconsumed and the next normal message wiped it
silently.
Fix: /confirm slash handler now returns a new "drive_confirm_turn"
action. run_repl detects that action and drives a real _drive_turn
with the synthetic "(operator typed /confirm — proceed with the
prior destructive proposal now; call the tool)" prompt. Same shape
as the browser cockpit's slash-command interceptor in
routes/cockpit.py around line 755. The /confirm in REPL now
actually fires the action.
2. **Gemma's "Done." wrap-up after a destructive BLOCK was misleading.**
When the LLM emitted a real tool_call for subscription.terminate,
the wrapper BLOCKED, the cockpit staged pending_destructive (and
printed "Pending /confirm for ..."), but Gemma's terminal AIMessage
was "Done." (v0.19 one-short-sentence rule picked the wrong words).
Operator sees "Pending /confirm for subscription.terminate" AND
"Done." in the bubble — looks like the action already ran.
Fix: when last_tool_proposal is set (= a destructive returned
BLOCKED this turn) AND the bubble is short (< 40 chars, i.e. a
wrap-up rather than a real explanation), replace the bubble with
an explicit "Proposed <tool>(args). Type /confirm to authorise."
The persisted transcript carries the right state; the next turn's
rehydration won't feed Gemma its own misleading "Done." Same fix
landed in both portals/csr cockpit route AND cli/bss_cli/repl.
Both fixes preserve pre-v1.5 behaviour for non-destructive turns —
the override only fires when last_proposal is set (= a destructive
BLOCKED was observed), and the new /confirm flow only triggers when
there's actually a pending row.
Verified:
- make test: orchestrator 420, cli 127, bss-cockpit 95, all green
- make e2e (rebuilt portal-csr): 12 passed, 1 skipped, 0 failed
in 40.04s. v15 specs unaffected.
REPL retest now: typing "terminate subscription SUB-0005" stages
pending, bubble reads "Proposed subscription.terminate(subscription_id=
'SUB-0005'). Type /confirm to authorise." Typing /confirm drives a
real turn — terminate actually executes — and the next "show
subscription SUB-0005" shows state=terminated.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…e post-BLOCK
Third-pass observation: Gemma can produce a 79-char "I propose to
terminate SUB-0005. Please type /confirm to proceed." narration that
PASSES the < 40 wrap-up gate (long enough to look like real prose)
AND survives the mimicry-stall warning (real tool_call did fire this
turn, so captured_tool_calls is non-empty). Result: the operator sees
honest "Pending /confirm" status above + misleading narrated propose
below, then types /confirm into a freshly-rebuilt REPL and... the
old code was loaded by their existing `bss` process anyway. Two
separate problems, same UX outcome.
Threshold removal:
Whenever last_tool_proposal is set (= a destructive returned
BLOCKED this turn), the bubble is replaced unconditionally with
the canonical "Proposed X(args). Type /confirm to authorise."
The pending_destructive row IS the truth; the bubble must match.
Trade-off: lose any legitimate prose Gemma might have added
("This will stop all data charges immediately"). The cost of
occasionally dropping that vs. the cost of misleading the
operator about what state the system is in is one-sided. Same
fix in both REPL and cockpit route.
The REPL-process-cache issue is unrelated to my code — Python
imports cache per-process; the operator was running a `bss` that
loaded the prior commit before /confirm-drives-turn landed. Restart
of `bss` is required between code changes, not a server-rebuild
issue. Document this in the operator-retest instructions next time.
make test green: orchestrator 420, cli 127, bss-cockpit 95.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…of "Done."
Live retest showed the workflow WORKING end-to-end (state went to
TERMINATED after /confirm) but the post-execute bubble was the same
"Done." word Gemma uses for every successful tool call. After the
operator just SAW "Done." on a stalled-propose path (where it was a
lie), seeing "Done." again — even though THIS time the action really
fired — is exactly what teaches them not to trust the cockpit.
The propose-side already overrides Gemma's wrap-up with the canonical
"Proposed X(args). Type /confirm to authorise." This commit lands the
symmetric override for the execute side.
Track destructive tool_calls that ACTUALLY RAN this turn (wrapper
let them through, result wasn't BLOCKED) as the new
executed_destructive list, parallel to last_tool_proposal /
last_proposal. After the turn's FinalMessage:
- If last_proposal set (propose path): "Proposed X(args). Type /confirm ..."
- Elif executed_destructive (execute path): "Executed X(args)."
(or for compound under batched, "Executed N destructive actions,
last was X(args).")
- Else: Gemma's wrap-up (the v0.19 anti-duplication rule keeps it short)
So a /confirm-resumed turn now produces a bubble that names what
actually happened:
> /confirm
/confirm staged for subscription.terminate args={'subscription_id': 'SUB-0005'}
╭─ bss ai ──────────────────────────────────────────────────────────────╮
│ Executed subscription.terminate(subscription_id='SUB-0005'). │
╰───────────────────────────────────────────────────────────────────────╯
Same fix landed in both REPL and cockpit route.
make test green: orchestrator 420, cli 127, bss-cockpit 95.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Single-line release version bump — picked up by every surface that reads the constant (REPL banner, self-serve portal brand-tag, CSR cockpit health, every service /health endpoint). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v1.5 unlocks multi-step orchestration in the cockpit — operators can ask compound questions ("investigate CASE-…", "register CUST + create order", "show customer and their subscriptions") and the agent chains the work. Adds
BSS_REPL_LLM_AUTONOMY={granular,batched}to control how many/confirms a compound destructive action needs. No new container, no schema, no migration (head stays at 0030).Design lifted from the proven loyalty-cli v0.11 pattern; adapted for bss-cli surfaces (REPL + browser veneer).
Five pieces
BSS_REPL_LLM_AUTONOMYenv var (orchestrator/bss_orchestrator/autonomy.py).granular(default — re-gates after each destructive) /batched(one/confirmauthorises the loop). Fail-closed at boot withAutonomyMisconfigured— same shape asBSS_API_TOKEN=changeme. Per-process scope; per-session override deferred to v1.5.1.Autonomy-aware destructive gating (
safety.py+graph.py+session.py).wrap_destructivegrowsautonomy_mode+loop_statekwargs;build_toolscreates oneLoopStateperbuild_graphinvocation and shares it across every destructive wrapper. Granular re-blocks after the first destructive fires; batched authorises the whole loop.DESTRUCTIVE_TOOLSunchanged — autonomy controls how many/confirms, not which tools.ITERATIVE FLOW prompt block + softened v0.19 "Done." rule (
bss_cockpit/prompts.py). Agent may chain another tool call instead of the mandatory terminating sentence — anti-duplication rule unchanged. Three bss-shaped worked examples (read chain, case investigation, compound write). Doctrine guard pinsITERATIVE FLOWto operator surface only — must not leak intocustomer_chat_prompt.py.3-strike loop bail (
session.py).MAX_CONSECUTIVE_TOOL_FAILURES=3terminatesastream_oncewith a structuredAgentEventErrorwhen the LLM keeps re-firing failure-shaped tool calls (real exceptions ORPOLICY_VIOLATION/DESTRUCTIVE_OPERATION_BLOCKED/CLIENT_ERROR). Counter resets on any success.Cockpit chrome filter (
bss_cockpit/chrome_filter.py). Strips cockpit-emitted chrome from history rehydration AND live emits — including the narrated-tool.name(args)mimicry shape Gemma produces. Plus inline post-propose / post-execute bubble overrides so/confirm-shaped bubbles say what they mean ("Proposed X(args). Type /confirm to authorise."/"Executed X(args).") instead of Gemma's misleading"Done.".Real-world bug fixes
Two pre-existing bugs surfaced + fixed during live REPL testing:
not allow_destructive_this_turnheuristic — silently lost the granular re-gate signal on/confirm-resumed turns. Rewired to derive strictly from COMPLETED-with-BLOCKED in both surfaces./confirmslash command was a no-op marker (returned a never-consumed string). Now drives a real_drive_turnwith the synthetic prompt the cockpit browser veneer always used.Doctrine
MAX_CONSECUTIVE_TOOL_FAILUREScontract,_ASSISTANT_CHROME_PREFIXESinventory lock.Test plan
make test— orchestrator 403→420 (+17 new unit tests across autonomy/safety/loop_bailout/iterative_flow_scope), bss-cockpit 66→95 (+29 chrome filter tests pinning every observed mimicry shape), cli 127 unchanged. All other suites unchanged. No regressions.make e2e(granular default) — 12 passed, 1 skipped (batched gated on env), 0 failed. New specstest_v15_compound_action_granular+test_v15_case_investigationgreen. All v1.4.1 specs unaffected.make e2e-batched(env-flipped) — 12 passed, 1 skipped (granular gated), 0 failed. Newtest_v15_compound_action_batchedspec green; granular spec auto-skips with a clear reason.terminate subscription→ propose →/confirm→ execute → verify state=TERMINATED workflow end-to-end. Bubble overrides ("Proposed X(args)…" / "Executed X(args)…") render correctly./confirm-into-the-void shows the mimicry-stall warning. Anti-mimicry runtime strip catches Gemma's"I propose to terminate ... \subscription.terminate(...)`"` shape.🤖 Generated with Claude Code