• Qwen-FAST: Utilizes Qwen2.5-VL-3B with a fast tokenizer to autoregressively generate discrete action tokens conditioned on visual and linguistic inputs (in line with π₀-fast).
• Qwen-OFT: Combines Qwen2.5-VL-3B with an MLP action head to perform parallel decoding of continuous actions, regressed from the hidden states of predefined special action tokens (in line with OpenVLA-OFT/EO).
• Qwen-PI: Integrates the Flow-Matching (FM) action expert with Qwen2.5-VL-3B, adopting a diffusion-based approach for continuous action prediction (in line with π₀).
• Qwen-GR00T: Implements a dual-system VLA architecture, where Qwen2.5-VL-3B serves as System2 for high-level vision-language reasoning, while the Flow-Matching module acts as System1 for rapid action prediction (in line with GR00T).

For dynamic updates, see our 🍀 Overleaf, which continuously presents our real-time experimental results.

We release a series of pretrained models and checkpoints to facilitate reproduction and downstream use.

• SimplerEnV
• LIBERO
• Robocasa
• RLBench
• RoboTwin
• BEHAVIOR

• Single Imitation Learning
• Multimodal Multitasks Co-training
• Reinforcement Learning Adaption

StarVLA emphasizes a modular model design. Each major framework file can be run standalone for rapid debugging and smoke test your code. For example:

# model
python starVLA/model/framework/QwenOFT.py --config_yaml internvla_cotrain_oxe.yaml
# dataloader
python starVLA/dataloader/lerobot_datasets.py --config_yaml internvla_cotrain_oxe.yaml

Note: starVLA/model/framework/yourframework.py is the single external API surface of the model; it should mirror (be structurally isomorphic to) the framework diagram in your paper.

StarVLA follows top‑down decomposition and the principle of high cohesion & low coupling.

For example:

• Dataloader
  • Returns a raw, model‑agnostic dict only; no model‑specific preprocessing (e.g., tokenizer, image encoding).
  • A single sample should include (add/remove as needed):
    • image: list[PIL.Image] | np.ndarray
    • lang: str
    • action: np.ndarray[T, action_dim]
    • state: Optional[np.ndarray[..., state_dim]]

Both framework.forward() and framework.predict_action() operate directly on raw inputs, keeping train/test boundaries explicit and easy to hack.

StarVLA uses a single global configuration object Parameters are passed primarily via extensible dicts, allowing overrides and controlled redundancy.

# check framework with fake examples
python starVLA/model/framework/QwenFM.py

It should build successfully and print(model). You can also call model.forward(fake_data) and obtain unnormalized actions via model.predict_action(fake_data).

The evaluation pipeline is adapted from InternVLA-M1

We also provide a parallel evaluation script:

check_pt=0723_v6_vla_dino_32/checkpoints/steps_10000_pytorch_model.pt
bash examples/SimplerEnv/eval_scripts/star_bridge.sh ${check_pt}

Before running, edit these variables directly at the top of star_bridge.sh.

Our training pipeline follows InternVLA-M1.

Steps:

1. Prepare a LeRobot-format OXE dataset, including modality.json. Refer to GR00T N1.5.
2. Add your dataset path to config.yaml:

   datasets:
     vla_data:
       dataset_py: lerobot_datasets
       data_root_dir: playground/Datasets/OXE_LEROBOT_DATASET  # path to your dataset
       data_mix: bridge_rt_1

3. Run with Accelerate:

   base_vlm=Qwen/Qwen2.5-VL-3B-Instruct
   Framework_name=QwenFM
   run_root_dir=./results
   run_id=${Framework_name}
   
   accelerate launch \
     --config_file starVLA/config/deepseeds/deepspeed_zero2.yaml \
     --num_processes 8 \
     starVLA/training/train_internvla.py \
     --config_yaml ./starVLA/config/training/internvla_cotrain_oxe.yaml \
     --framework.framework_py ${Framework_name} \
     --framework.qwenvl.base_vlm ${base_vlm} \
     --run_root_dir ${run_root_dir} \
     --run_id ${run_id} \
     --wandb_project your_project \
     --wandb_entity your_name

Note: run_root_dir stores the unified config snapshot and data‑processing metadata for reproducibility and quick restarts.

A: We profiled it: data preprocessing takes <1% time. Keeping it inside the Framework is acceptable and allows model‑specific flexible handling.

A: Yes. Implement new vision + language modules and compose them inside a Framework; any other existing models can be swapped in. yet, due to framework precessing raw action data, it is very easy to swapped in.

A: Yes. We use OmegaConf.load(args.config_yaml) as the single configuration entry; standalone debugging also uses args.config_yaml. Parameters may be intentionally redundant; you can freely add or override them via the CLI.

Examples:

accelerate launch \
  --config_file starVLA/config/deepseeds/deepspeed_zero2.yaml  \
  --num_processes 8 \
  starVLA/training/train_internvla.py \
  --config_yaml ./starVLA/config/training/internvla_cotrain_oxe.yaml \
  --framework.qwenvl.base_vlm Qwen/Qwen2.5-VL-7B-Instruct \ # override framework choice
  --framework.qwenvl.base_vlm Qwen/Qwen2.5-VL-7B-Instruct \ # override framework choice
  --framework.action_model.new_module ${module_name} \ # plug-in a new module to action model

⚠️: framework.action_model.new_module only adds to the global config; its behavior is on your framework.

A: Yes. StarVLA uses a regex / name list to control freezing. Example:

--trainer.freeze_modules "qwen_vl_interface.model.model.visual,dino_encoder" \

Tips: You can print(your_model) first to check the relative paths of your modules and list them as comma-separated values. (implementation in TrainerUtils.freeze_backbones.)

A: Yes, starVLA also uses name: value dict to control learning group. Config example:

trainer:
  learning_rate:
    base: 1e-05      # other modules
    qwen_vl_interface: 1.0e-05
    action_model: 1.0e-04

(Also referenced in trainer_tools.build_param_lr_groups.)

A: Yes, somehow can. Specify the latest checkpoint path in config.yaml, e.g.:

trainer:
  pretrained_checkpoint: path_to_steps_10000.pt
  reload_modules: "action_model"

Empty reload_modules means full load all model. However, starVLA does not save optimizer state. It requires a lot of memory/disk and bring limited benefit.