feat(envs): SallySoprano — LLM-judged salary negotiation game - #202
Open
borgr wants to merge 6 commits into
Open
feat(envs): SallySoprano — LLM-judged salary negotiation game#202borgr wants to merge 6 commits into
borgr wants to merge 6 commits into
Conversation
- Implement 2-player bilateral negotiation game with LLM judge evaluation - Add complete environment with proposal/acceptance mechanics and automatic draw - Include robust action parsing, invalid move handling, and round management - Add full test suite (18 tests) covering all game mechanics and edge cases
The environment shipped without a register_with_versions entry, so
ta.make("SallySoprano-v0") — the call used in the env's own README and
examples — raised. Register it alongside the other negotiation role-plays
(3 players: two negotiators + LLM judge, max_rounds=60).
…moves, drop dead reward writes - Pass error_allowance through to FFAMultiPlayerState (was declared but unused; state defaulted to 1). - Repeated invalid moves now forfeit to the opponent instead of leaving the offender stuck on their turn, which could hang the game against a persistently malformed agent. - step() reports done correctly after a forfeit. - Remove self.rewards writes shadowed by state.set_winners/set_draw, and a redundant local import re.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebases and finishes @cstorm125's #167 against current
main.What it is
The classic Sally Soprano negotiation exercise as a 3-player
FFAMultiPlayerStategame: Player 0 = Sally's Agent, Player 1 = the Lyric Opera's Business Manager, Player 2 = an LLM judge who scores the outcome once a deal is struck (or max rounds elapse). Each side gets confidential instructions; they exchange free text and[Propose] <amount>/[Accept].Changes on top of #167
SallySoprano-v0(+-train).error_allowanceinto the state — it was declared on the env (and asserted in the tests) but never passed through, so the state silently used its default of 1.elimination_orderbut the env ignored it, leaving that player stuck on their turn — a persistently malformed agent could hang the game since only valid moves advance the round counter.step()now also reportsdonecorrectly after a forfeit.self.rewards = {...}writes that were shadowed bystate.set_winners/set_draw(dead —close()readsstate.rewards), plus a redundant localimport re.Verification (no CI — all local)
max_roundswith no deal → automatic draw{0:0,1:0,2:0}, terminates.[Propose]→ exhausts allowance → forfeits, opponent+1, offender-1, terminates.Verification gap (called out honestly)
The full deal→judge→reward path needs a real model acting as Player 2 (the judge). I verified the judge's parser and reward mapping directly and via the author's canned-decision tests, but did not run a live model as judge. Everything not requiring the judge is exercised end-to-end above.
Credit to @cstorm125 (#167) for the design and implementation.