Add deepseek 3.2 exp#41251
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
Hello, thanks for your support for deepseek v3.2! I wonder when this PR will be ready? |
|
Working on it! Hoping by next week 🤗 |
|
wow this got old! |
|
The submitted code is currently unusable and does not support the Deepseek-v3.2 official version. Is this PR still being updated? |
https://github.com/yunkchen/transformers/tree/v4.57.3_add_dpskv32 |
There is stillDeepseekV32Attention a bug when using LLMC to quantify the model:[rank0]: Traceback (most recent call last): |
New commit pushed, sorry. |
A bug still occurred when running the LLMC quantization model: [rank0]: Traceback (most recent call last): |
I'm seeing problems in this branch with rope factor/beta_fast/beta_slow values not being floats. Is this an oversight? |
Isn't this implementation still O(L^2) since it just masks full attention to the indexer's topk? |
|
@ArthurZucker Is this ready to merge? I'd really love to experiment with some DeepSeek 3.2 Speciale fine tunes. |
|
@ArthurZucker @yunkchen |
|
@ArthurZucker Happy holidays, checking in again :) Can we get this merged please? |
|
Hey! Thanks, just got back from holidays, we shipped https://github.com/huggingface/transformers/blob/57278c904c5158999d31a0db8bfcd63360c37b48 but now I should be able to get back! sorry for the delay everyone v5 needed a slow down in model addition to support all the new features, especially default FP8 weight support! |
|
Thanks @ArthurZucker, do you have an ETA? Getting this in would be massively helpful to me and the community. Happy to help however I can. |
|
Hi @ArthurZucker, just checking in to see if there are any updates on this? There is a lot of interest for this change, so we’re excited to see it move forward! :) Thanks! |
|
Hey all! Yeah I am mega underwater! shipping v5 took / takes longer than we thought, but its still planned. Hopefully for end of week! really sorry its very vague but yeah should be alright |
|
UPDATE: #44054 is gonna take a bit of time so I'll finish this one first |
|
Hi @ArthurZucker, Checking if there is any new updates. Look forward to the DeepSeek3.2 feature! |
…s into add-deepseek-exp
…s into add-deepseek-exp
|
run-slow: auto, deepseek_v3, deepseek_v32, glm4_moe_lite, glm_moe_dsa, longcat_flash, mistral4, youtu |
|
This comment contains models: ["models/auto", "models/deepseek_v3", "models/deepseek_v32", "models/glm4_moe_lite", "models/glm_moe_dsa", "models/longcat_flash", "models/mistral4", "models/youtu"] |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: auto, deepseek_v3, deepseek_v32, glm4_moe_lite, glm_moe_dsa, longcat_flash, mistral4, youtu |
DeepSeek-V3.2-Exp ships config.json with model_type "deepseek_v32" and no auto_map. On transformers releases without native deepseek_v32 support (our >=4.56 floor still admits these), the AutoConfig lookup that AutoTokenizer performs internally fails and tokenizer loading aborts before any benchmark traffic. --tokenizer-trust-remote-code cannot help because, with no auto_map, there is no remote config class to import. Register a narrow DeepseekV3Config alias under model_type "deepseek_v32" before loading the tokenizer (V3.2 reuses the V3 config schema; vLLM and SGLang do the same). The shim is idempotent and best-effort: a no-op when transformers already knows the model type natively, and silent when the base config class is unavailable so loading reaches its normal error path. Native support landed upstream in huggingface/transformers#41251; this covers the older releases in our supported range that predate it. Fixes ai-dynamo#1047 Signed-off-by: Siwei Li <[email protected]>
What does this PR do?