Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add old InternVL2-1B/2B support to the InternVL conversion script #45092#45097

Open
baonudesifeizhai wants to merge 2 commits into
huggingface:mainfrom
baonudesifeizhai:fixinternvl2
Open

Add old InternVL2-1B/2B support to the InternVL conversion script #45092#45097
baonudesifeizhai wants to merge 2 commits into
huggingface:mainfrom
baonudesifeizhai:fixinternvl2

Conversation

@baonudesifeizhai

Copy link
Copy Markdown

What does this PR do?

This PR extends the InternVL conversion script to support the old OpenGVLab/InternVL2-1B and OpenGVLab/InternVL2-2B checkpoints.
These checkpoints currently rely on remote code and are problematic for downstream users on Transformers v5. Instead of instantiating the original remote-code models, the converter now reads the original config and weights directly and emits HF-native InternVLForConditionalGeneration checkpoints.

Fixes # (issue)
#45092
38425

before that : on vllm main branch:
pytest
tests/models/multimodal/generation/test_common.py
-k 'intern_vl2-hf-local and test_multi_image_models' -vv broken;

this branch:

VLLM_TEST_INTERNVL2_HF_MODEL=/tmp/InternVL2-1B-hf /root/venv/bin/python -m pytest
tests/models/multimodal/generation/test_common.py
-k 'intern_vl2-hf-local and test_multi_image_models' -vv passed

Validation

Ran the following with source ~/venv/bin/activate:

python src/transformers/models/internvl/convert_internvl_weights_to_hf.py \
  --input_dir OpenGVLab/InternVL2-1B \
  --output_dir /tmp/InternVL2-1B-hf

python src/transformers/models/internvl/convert_internvl_weights_to_hf.py \
  --input_dir OpenGVLab/InternVL2-2B \
  --output_dir /tmp/InternVL2-2B-hf

- [ ] I confirm that this is not a pure code agent PR.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc
- kernels: @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->

@ArthurZucker ArthurZucker left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to have!
Dynamic Weight Loader + attribute map + TokenizersBackend could potentially work OOB as well? if we add the model type to

TOKENIZER_MAPPING_NAMES = OrderedDict[str, str | None](
it will skip remote code for the tokenizer at least

@hmellor

hmellor commented Mar 31, 2026

Copy link
Copy Markdown
Member

I don't think we can use this to solve the vLLM side issue because it requires recreating the checkpoints

@baonudesifeizhai

Copy link
Copy Markdown
Author

k i will try t dynamic Weight Loader + attribute map + TokenizersBackend...

I don't think we can use this to solve the vLLM side issue because it requires recreating the checkpoints

@baonudesifeizhai baonudesifeizhai force-pushed the fixinternvl2 branch 2 times, most recently from 9b9c427 to d83c6be Compare April 1, 2026 02:19
@baonudesifeizhai

Copy link
Copy Markdown
Author

PYTHONPATH=/root/zdj/transformers/src
VLLM_TEST_INTERNVL2_HF_MODEL=/tmp/InternVL2-1B-hf
python -m pytest tests/models/multimodal/generation/test_common.py
-k 'intern_vl2-hf-local and test_single_image_models'
-vv -x -s

and

PYTHONPATH=/root/zdj/transformers/src
VLLM_TEST_INTERNVL2_HF_MODEL=/tmp/InternVL2-1B-hf
python -m pytest tests/models/multimodal/generation/test_common.py
-k 'intern_vl2-hf-local and test_multi_image_models'
-vv -x -s

passed on vllm

@baonudesifeizhai

Copy link
Copy Markdown
Author

@github-actions

github-actions Bot commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, internvl

@baonudesifeizhai

Copy link
Copy Markdown
Author

python -m pytest tests/models/multimodal/generation/test_common.py::test_single_image_models -vv -s -x -k "intern_vl-test_case25 or intern_vl-test_case26 or intern_vl-test_case27 or intern_vl-test_case28 or intern_vl-test_case29 or intern_vl-test_case30" passed on vllm side

@baonudesifeizhai

Copy link
Copy Markdown
Author

https://paste.ubuntu.com/p/wJWpJwgvst/ passed python -m pytest tests/models/multimodal/generation/test_common.py::test_single_image_models -vv -s -x -k "intern_vl-test_case25 or intern_vl-test_case26 or intern_vl-test_case27 or intern_vl-test_case28 or intern_vl-test_case29 or intern_vl-test_case30"

@github-actions

github-actions Bot commented Apr 2, 2026

Copy link
Copy Markdown
Contributor

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=45097&sha=da3790

@baonudesifeizhai

Copy link
Copy Markdown
Author

god will fix it tomorrow...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants