Fixed a bug in 15puzzle and updated logic for Wordle and 15puzzle + Sample Multilingual code - #169

Hey @MadBonzz , thanks for fixing that issue in 15 puzzle. Random swapping N times, based on difficulty, from the ordered outcome makes total sense. Multilingual is a great suggestion too. Just not sure if adding a player prompt file for every environment is the most efficient way - we have 100 environments and counting hahah. How about writing a translation module that uses a model from huggingface? Nice touch on Wordle.

Hey @bobbycxy I understand adding separate files for each game is quite redundant, but I don't think using a module is a good idea, the translation quality is not good enough, and it might be slower to do on the fly. The translation quality really matters because bad translation can put the model at a disadvantage in a language, making the evaluation unfair.

Alternatively, we can probably use some model on OpenRouter itself to translate, have a translation prompt set up, and then use it to get the translation. Will have to implement caching to save credits, but having files with translations seemed the best idea in terms of ensuring translation quality and speed.

I also wanted to discuss on this:

• Multi-lingual envs Wordle, Codenames, GuessWho etc would require translations of the words/characters. How do you wish to go about this? I was planning to have translation stored itself, like with the prompts.
• Also need to update the envs to allow users to use their own word lists/jsons etc.

great idea adding additional languages! Looks good!

More broadly speaking, what would you think about an Observation Wrapper that takes care of the translation (i.e. adding one here (https://github.com/LeonGuertler/TextArena/blob/main/textarena/wrappers/ObservationWrappers/llm_observation_wrapper.py) that will take care of the translation into arbitrary languages; might have the added advantage that no additional code changes for new environments are necessary.

Maybe something along these lines:

from google.cloud import translate_v2 as translate

class GoogleLLMObservationWrapper(ObservationWrapper):
    def __init__(self, env: Env, target_lang: str = "en"):
        super().__init__(env)
        self.full_observations: Dict[int, List[Tuple[int, str]]] = {}
        self.client = translate.Client()
        self.target_lang = target_lang

    def _convert_obs_to_str(self, player_id: int) -> Observations:
        str_observation = ""
        if player_id in self.full_observations:
            for sender_id, message, _ in self.full_observations[player_id]:
                if sender_id == ta.GAME_ID: sender_name = "GAME"
                else:                       sender_name = self.env.state.role_mapping.get(sender_id, f"Player {sender_id}")
                str_observation += f"\n[{sender_name}] {message}"
        return str_observation

    def observation(self, player_id: int, observation: Optional[ta.Observations]):
        if observation is None: return self._convert_obs_to_str(player_id=player_id)
        if player_id not in self.full_observations: self.full_observations[player_id] = []

        # translate each incoming observation's message (index 1) and preserve tuple shape
        translated = []
        for obs in observation:
            sender = obs[0]
            msg = "" if obs[1] is None else str(obs[1])
            try:
                res = self.client.translate(msg, target_language=self.target_lang)
                msg_tr = res.get("translatedText", msg)
            except Exception: msg_tr = msg  # on failure, keep original
            rest = tuple(obs[2:]) if len(obs) > 2 else ()
            translated.append((sender, msg_tr) + rest)
        self.full_observations[player_id].extend(translated) # Append translated observations in sequence
        return self._convert_obs_to_str(player_id=player_id)

Might have the additional advantage of allowing models to play games like Negotiation or SecretMafia in different languages. What do you think?

Very happy to merge this already, the wrapper is just a suggested in the long-run.

To summarize the meet,

• we're doing both google translate and openrouter, we will let the user choose
• for openrouter, will look for models good at translation. In my experience, gemini models have been good. Gemini-flash can be an affordable option

I'll try to make the updated PR in 2-3 days.

Added translation feature but not tested yet.
Do not merge right now.
Apologies for the delay.

Updated and fixed some issues with translation wrapper. Tested the OpenRouter version, it is working correctly.
Currently have made it such that TranslationWrapper needs to be applied manually, by creating the raw version of the env first.
Ready to be merged.
Let me know if you want me to make changes to the envs init.py to make it one of the default wrappers.

The solvability idea here is genuinely valuable — as-is on main, _generate_board uses a uniform random.shuffle, so ~50% of FifteenPuzzle episodes are unsolvable (measured: 9983/20000 solvable). That's worth landing on its own, so I've split just that fix into a small, verified standalone PR: #196 (single-file change, 0/20000 unsolvable after the fix). It can merge independently and immediately.

For this PR to move as-is, a few things still block it — flagging so you can decide how to proceed:

• ta.make("FifteenPuzzle-v0") would raise TypeError. The registry (textarena/envs/__init__.py) passes max_turns=200 to the constructor, but the new __init__(self, difficulty=...) drops the max_turns parameter. This PR doesn't update the registration, so the env no longer constructs.
• Scope is bundled. Alongside the puzzle change it also rewrites Wordle's constructor (word_length/num_guesses → word_lengths), adds two Google-Translate observation wrappers into the wrapper registry, and adds a player_prompts.py/utils.py. Each of those is a separate concern and would be easier to review/land as its own PR.

Happy to help rebase the Wordle piece separately if useful. Not trying to step on the work — just unblocking the solvability win so it isn't held up by the rest.

Thanks for this, @MadBonzz — three separate improvements are bundled here, so I reviewed each against main. Up front: the solvability fix and the Wordle logging fix are both worth landing and have been done as focused PRs (#196, #186); the difficulty / variable-length redesigns and the translation wrapper each need a different shape than what's here.

1. FifteenPuzzle — solvability + difficulty

The solvability intent is exactly right: main's random.shuffle can produce unsolvable boards. That's fixed non-breakingly in #196 (parity check + single swap; keeps the max_turns constructor and the FifteenPuzzle-v0 registration).

The difficulty redesign can't merge as-is:

• utils.generate_puzzle raises TypeError on every reset — find_blank matches board[i][j] == 0, but the blank tile is None (list(range(1,16)) + [None]), so it returns None and available_moves(None) fails to unpack.
• The outer for n_swaps in range(1, swaps+1) loop rebuilds the board from solved each pass, so only the last iteration has any effect.
• __init__(difficulty=...) drops max_turns, which breaks the registered FifteenPuzzle-v0 (max_turns=200).
• player_prompts.py is added but never imported by env.py, so those translations are currently dead code.

Difficulty tiers are a nice idea — best done later as an additive (non-breaking) option on top of the #196 generator.

2. Wordle — variable word length

main already exposes multiple lengths via registered variants (Wordle-v0 = 5 letters, Wordle-v0-long = 7, each ±-hardcore), which keeps every env id at a fixed, reproducible difficulty. The word_lengths: List[int] redesign here would:

• crash at construction — _load_word_list() still reads self.word_length, which the new __init__ no longer sets (AttributeError);
• break all four registered variants, which pass word_length= / num_guesses= kwargs the new constructor rejects;
• pick a random length per reset, making a single env id's difficulty non-deterministic across episodes.

The one Wordle bug worth fixing — the player's guess logged as coming from GAME — is addressed in #186 (from_id=player_id).

3. Observation translation wrapper

GoogleTranslationWrapper (Google Cloud Translate) and OpenRouterTranslationWrapper (OpenRouter) translate observations at runtime. That's a real feature, but it's a separate concern from the two game fixes and pulls in external services/credentials. If it's wanted, a focused standalone PR — with the deps made optional so importing the wrapper package doesn't require them — would be the way to evaluate it on its own merits.

Recommendation: close in favor of #196 (solvability) and #186 (Wordle logging), which deliver both fixes without the breaking API changes. Happy to help shape the difficulty-tier option or the translation wrapper as their own PRs if you'd like them. Thanks again for surfacing both the solvability and logging bugs — they're getting fixed because of this.

Hey

I won't be able to do the translation wrapper PRs at the moment so I will be closing the PR.
Happy to have been of help in identifying the issues.