🌟 This is the official repository which contains the training and inference code for ThinkMorph.

• [2025.12.22] The evaluation code for ThinkMorph is now accessible at VLMEvalKit_Thinkmorph.
• [2025.10.29] Our model checkpoint and training data are now accessible at Huggingface.
• [2025.10.29] Our paper is now accessible at arxiv.

We present ThinkMorph, a unified model fine-tuned on ∼24K high-quality interleaved reasoning traces across tasks, learning to generate progressive text–image reasoning steps that concretely manipulate visual content while maintaining coherent verbal logic.

Beyond strong vision-benchmark performance and robust out-of-domain generalization, ThinkMorph demonstrates emergent multimodal intelligence, including novel visual manipulation skills and so on. These findings suggest promising directions for characterizing the emergent capabilities of unified models for multimodal reasoning.

1️⃣ Set up environment

git clone https://github.com/ThinkMorph/ThinkMorph.git
cd ThinkMorph
conda create -n thinkmorph python=3.10 -y
conda activate thinkmorph
pip install -r requirements.txt

2️⃣ Download checkpoint

from huggingface_hub import snapshot_download

save_dir = "models/ThinkMorph-7B"
repo_id = "ThinkMorph/ThinkMorph-7B"
cache_dir = save_dir + "/cache"

snapshot_download(cache_dir=cache_dir,
  local_dir=save_dir,
  repo_id=repo_id,
  local_dir_use_symlinks=False,
  resume_download=True,
  allow_patterns=["*.json", "*.safetensors", "*.bin", "*.py", "*.md", "*.txt"],
)

3️⃣ Use inference.ipynb to play with ThinkMorph!

We opensource our training data mentioned in our paper containing four tasks: Jigsaw Assembly, Spatial Navigation, Visual Search , and Chart Refocus. Here we show typical examples of four tasks. Training data can be downloaded from Huggingface.

1. Download the training dataset

    from datasets import load_dataset
   
    # Jigsaw Assembly
    dataset = load_dataset("ThinkMorph/Jigsaw_Assembly", split="train")
   
    # Spatial Navigation
    dataset = load_dataset("ThinkMorph/Spatial_Navigation", split="train")
   
    # Visual Search
    dataset = load_dataset("ThinkMorph/Visual_Search", split="train")
   
    # Chart Refocus
    dataset = load_dataset("ThinkMorph/Chart_Refocus", split="train")

2. Convert the downloaded dataset into a data format suitable for model training. For details on the Bagel officially supported data formats, see in Train. Based on Bagel's implementation, we modify the training code to support our interleaved data format, and an easy-to-understand example of a parquet file is shown below:

{
    "image_list": [problem_image_0, reasoning_image_0],
    "instruction_list": [question],
    "output_text_list": [f"<think>{resoning_thought_0}</think><image_start>",f"<image_end><think>{resoning_thought_1}</think><answer>{answer}</answer>"],
}

3. Edit data/dataset_info.py with your own data path.

4. Edit configs/example.yaml. Additionally, we provide example configuration files corresponding to the different training settings in data/configs.

torchrun \
  --nnodes=$num_nodes \
  --node_rank=$node_rank \
  --nproc_per_node=8 \
  --master_addr=$master_addr \
  --master_port=$master_port \
  train/pretrain_unified_navit.py \
  --dataset_config_file ./data/configs/interleaved_reasoning.yaml \
  --model_path $model_path \
  --layer_module Qwen2MoTDecoderLayer \
  --finetune_from_hf True \
  --auto_resume True \
  --finetune-from-ema True \
  --resume-from $model_path \
  --results_dir $output_path \
  --checkpoint_dir $ckpt_path \
  --lr 1e-5 \
  --num_worker 4 \
  --max_latent_size 64  \
  --max_num_tokens 32768 \
  --mse_weight 1 \
  --ce_weight 1 \
  --total_steps 8000 \

You can replace the variables in the script with your own before running. More training scripts are provided in ./script. See Bagel's TRAIN for more details.

Our evaluation code is open-sourced in VLMEvalKit_Thinkmorph. This repository provides evaluation support for the ThinkMorph model based on VLMEvalKit. And this repo also supports all the benchmarks evaluated in our paper, including: VSP, VisPuzzle, ChartQA, VStar, BLINK-J, MMVP, SAT, BLINK, and CV-Bench.

@article{gu2025thinkmorph,
  title={ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning},
  author={Gu, Jiawei and Hao, Yunzhuo and Wang, Huichen Will and Li, Linjie and Shieh, Michael Qizhe and Choi, Yejin and Krishna, Ranjay and Cheng, Yu},
  journal={arXiv preprint arXiv:2510.27492},
  year={2025}
}