fix(gguf): correct mismatched-shape error message in check_quantized_param_shape#13504
Conversation
|
@Ricardo-M-L I am seeing that you're opening a lot of PRs in a very short period of time. I politely as you to reduce that pace a bit. |
|
Friendly ping — this PR has been approved. Is there anything else needed before merging? Happy to make any requested changes. |
check_quantized_param_shape compares inferred_shape against current_param_shape, but the error message printed inferred_shape vs loaded_param_shape — and inferred_shape is derived from loaded_param_shape, so the reported mismatch was effectively self-referential and gave no signal about the model's expected shape. Print current_param_shape (what the model expected) vs inferred_shape (what the quantized weight decodes to) so the two sides of the comparison are actually visible. Noted by @Vargol in huggingface#13001.
be36bde to
587609f
Compare
|
Thank you for the feedback, @sayakpaul! I understand the concern about PR volume. I will be more mindful and focus on higher-impact contributions going forward. Sorry for the inconvenience. |
|
Hi @DN6 @sayakpaul, gentle bump — this one is approved (DN6) and mergeable; just hasn't been picked up yet. It's a one-line message-formatting fix so existing user-reported shape mismatches surface the actual |
|
@DN6 sorry for the second ping — this one was approved on Apr 23 and has been sitting If you've got a free moment, could you (or anyone with merge rights) hit the merge button? Happy to rebase on the latest main if needed 🙏 |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…param_shape (#13504) fix(gguf): correct mismatched-shape error message check_quantized_param_shape compares inferred_shape against current_param_shape, but the error message printed inferred_shape vs loaded_param_shape — and inferred_shape is derived from loaded_param_shape, so the reported mismatch was effectively self-referential and gave no signal about the model's expected shape. Print current_param_shape (what the model expected) vs inferred_shape (what the quantized weight decodes to) so the two sides of the comparison are actually visible. Noted by @Vargol in #13001.
What does this PR do?
Fixes the misleading error raised by
GGUFQuantizer.check_quantized_param_shapewhen a loaded GGUF weight doesn't match the model's expected shape.Before
The check compares
inferred_shapeagainstcurrent_param_shape, but the message reportsinferred_shapevsloaded_param_shape. Sinceinferred_shapeis derived fromloaded_param_shape, the two values on either side of the reported "mismatch" are effectively the same thing described at different unpacking stages — the shape the model actually expected (current_param_shape) never shows up in the message.Concretely, the 9B Q8 GGUF failure noted in #13001 produced:
…even though the model parameter was
(36864, 6144), which is the real expected shape and the thing the user needs to see when diagnosing a Klein-vs-Dev/GGUF-variant mix-up.After
Now both sides of the comparison are visible, and the "expected" side actually reflects what the model wants.
Related
Partially addresses the error-message confusion noted by @Vargol in #13001 (comment). This PR only touches the error text — it does not change the detection logic or attempt to resolve the underlying Klein-vs-Dev GGUF shape-inference issue that @DN6 is tracking.
Before submitting
Who can review?
@DN6 @sayakpaul