fix(draft): read RoPE parameters from Transformers v5 configs - #702
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a helper function get_rope_config to support both v4 and v5 RoPE configurations across llama3_eagle.py and peagle.py, ensuring robust extraction of rope_theta and rope_scaling. Corresponding unit tests have also been added to verify this behavior. The review feedback correctly identifies a potential KeyError or TypeError when directly indexing rope_params["rope_theta"] and suggests a safer fallback implementation.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| rope_params = getattr(config, "rope_parameters", None) | ||
| if rope_params is not None: | ||
| return rope_params["rope_theta"], rope_params | ||
| return getattr(config, "rope_theta", 10000), getattr(config, "rope_scaling", None) |
There was a problem hiding this comment.
Accessing rope_params["rope_theta"] directly can raise a KeyError if the key is missing from the dictionary, or a TypeError if rope_params is not a dictionary (e.g., if it is a custom configuration object). To ensure robustness across different Transformers versions and custom model configurations, we should safely retrieve rope_theta using .get() or getattr(), with a fallback to config.rope_theta or 10000.
| rope_params = getattr(config, "rope_parameters", None) | |
| if rope_params is not None: | |
| return rope_params["rope_theta"], rope_params | |
| return getattr(config, "rope_theta", 10000), getattr(config, "rope_scaling", None) | |
| rope_params = getattr(config, "rope_parameters", None) | |
| if rope_params is not None: | |
| if isinstance(rope_params, dict): | |
| rope_theta = rope_params.get("rope_theta") | |
| else: | |
| rope_theta = getattr(rope_params, "rope_theta", None) | |
| if rope_theta is None: | |
| rope_theta = getattr(config, "rope_theta", 10000) | |
| return rope_theta, rope_params | |
| return getattr(config, "rope_theta", 10000), getattr(config, "rope_scaling", None) |
|
I have tested the failed unittest, it's OK. |
Motivation
Transformers v5 refactored RoPE configuration in [huggingface/transformers#39847]. Starting from Transformers 5.0.0, RoPE parameters are standardized under
config.rope_parameters, andconfig.rope_thetais no longer guaranteed to exist.In Transformers 4.57.1, Llama configs stored the values as top-level attributes, configuration_llama.py.:
In Transformers 5.x, legacy values are moved into
rope_parametersduring config initialization, modeling_rope_utils.py:After SpecForge upgraded Transformers from 4.57.1 to 5.8.1, the custom EAGLE3 and P-EAGLE RoPE implementations still read the legacy top-level
config.rope_theta. For configs using a non-default RoPE base, this could silently fall back to10000and produce rotary embeddings inconsistent with the target model.Modifications
get_rope_confighelper compatible with both Transformers v4 and v5:config.rope_parameterswhen available.config.rope_thetaandconfig.rope_scalingattributes.rope_thetaand RoPE parameters without mutating the Transformers config object.self.rope_scalingso all MRoPE attention backends use the same configuration source.rope_thetapropagation in EAGLE3 and MRoPE.Checklist