Oucheng Huang1* Yuhang Ma2*† Zeng Zhao2✉ Mingrui Wu1
Jiayi Ji1 Rongsheng Zhang2 Zhipeng Hu2 Xiaoshuai Sun1✉ Rongrong Ji1
1 Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University
2 Fuxi AI Lab, NetEase Inc.
*Equal Contribution †Project Lead ✉Equal Advising
Follow the steps below to set up the environment:
-
Create a new conda environment and activate it:
conda create -n comfygpt python==3.10 conda activate comfygpt
-
Install the required dependencies:
pip install -r requirements.txt
-
Download Model and Resources from this link and place it in the
./comfygpt/directory. -
Inference
python infer.py --instruction "This workflow can generate image, using sd3 model."
To train our RefineAgent, we utilize LLaMA-Factory, a powerful framework for fine-tuning large language models.
The training configuration files, including the JSON and YAML files, can be found in the train/sft directory.
If you find Comfygpt useful in your research, please cite our work:
@misc{huang2025comfygptselfoptimizingmultiagentcomprehensive,
title={ComfyGPT: A Self-Optimizing Multi-Agent System for Comprehensive ComfyUI Workflow Generation},
author={Oucheng Huang and Yuhang Ma and Zeng Zhao and Mingrui Wu and Jiayi Ji and Rongsheng Zhang and Zhipeng Hu and Xiaoshuai Sun and Rongrong Ji},
year={2025},
eprint={2503.17671},
archivePrefix={arXiv},
primaryClass={cs.MA},
url={https://arxiv.org/abs/2503.17671},
}