Remove attribute_map from GptOssConfig#45578
Conversation
Added in huggingface#45473 but has no reader; it clobbers num_local_experts when checkpoints carry both keys (breaks tiny-GptOssForCausalLM loading in PEFT/TRL CI).
|
[For maintainers] Suggested jobs to run (before merge) run-slow: gpt_oss |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
cc @ArthurZucker @BenjaminBossan from the previous issue |
There was a problem hiding this comment.
As privately discussed with @AmineDiro, the previous issue has been independently fixed by:
- huggingface/trl#5622
- https://huggingface.co/trl-internal-testing/tiny-GptOssForCausalLM/discussions/2/files
The precedent PR #45473 just made evident this pre-existing bug. See explanation in huggingface/trl#5622 description.
In relation with this PR, and independently of the above mentioned issue, I think it is a transformers API/design decision: Whether an attribute mapping num_experts → num_local_experts in GptOssConfig makes sense, to align with Mixtral/MiniMax
- I would defer this decision to the maintainers of
transformers
|
My understanding is that this should no longer be necessary, but I'll leave it up to others to make the call. |
|
I can just close this PR if it's not needed. We just need to confirm API parity with Mixtral/MiniMax :) ? |
Added in #45473, it clobbers num_local_experts when checkpoints carry both keys (breaks tiny-GptOssForCausalLM loading in PEFT/TRL CI) : https://github.com/huggingface/transformers/pull/45473/changes/BASE..20c2a54ea0f7669d1c5af1fa512ca91379e48df6#r3116765952
Was added for API parity with Mixtral/MiniMax
{"num_experts": "num_local_experts"}. Looked like a natural consistency change to land alongside the EP fixes.Transformers CI didn't fail. The bug only fires when a
config.jsoncarries bothnum_expertsandnum_local_experts. GPT-OSS-20 has only the latter, so the transformers tests pass. TRL/PEFT CI pin ontrl-internal-testing/tiny-GptOssForCausalLM, whose config has both (num_experts: 4, num_local_experts: 128). With the attribute_map, the num_experts=4 kwarg silently rewrites num_local_experts to 4, so the built model has 4 experts while thecheckpoint holds 128, that's why we got the shape MISMATCH.
Although I saw @albertvillanova fixing the
num_local_experts👀 in https://huggingface.co/trl-internal-testing/tiny-GptOssForCausalLM/commit/ffe015e42676a3b43e8ff4534b67d3f3eeb0f25aWhat does this PR do?
Who can review?
@ArthurZucker @BenjaminBossan @qgallouedec @albertvillanova