MCoreAdapter is a lightweight bridge toolkit for scalable LLM/VLM training, combining NVIDIA Megatron-LM's distributed training efficiency with HuggingFace Transformers-like API simplicity.
Developed as Roll Framework's Megatron-LM integration layer, it enables seamless interoperability between Roll's reinforcement learning workflows and Megatron's distributed training capabilities.
pip install "git+https://github.com/alibaba/roll.git#subdirectory=mcore_adapter"Except reinforcement learning with Roll, MCoreAdapter can also be applied for LLMs and VLMs in PreTraining, SFT and DPO/ORPO.
See examples for fine-tuning examples used LLaMA-Factory library.
Convert a Megatron model to HuggingFace model:
python tools/convert.py --checkpoint_path path_to_megatron_model --output_path path_to_output_hf_modelMCoreAdapter can directly load a HuggingFace model, so you can skip converting the model to Megatron.