T2ITrainer is a diffusers based training script. It aims to provide simple yet implementation for lora training.

• ❗ Mandatory: Update diffusers to latest github version

pip install git+https://github.com/huggingface/diffusers.git -U

• 2025-12-20: Node Based Frontend UI for configuration with visualization capabilities. Flexible dataset configuration. (Still under development)
• 2025-12-20: Support LongCat Image and LongCat Edit, 6B MMDIT flux vae models, Lora Training

• PyTorch: torch>=2.3.0+cu121 (CUDA 12.1 supported)
• Node.js: node>=14.0.0 (Required for frontend UI)

❗ Mandatory: Install Microsoft Visual C++ Redistributable if encountering DLL errors

❗ Mandatory: Install Node.js (version 14 or higher) for the Node-Based Frontend UI

After installing Node.js, verify the installation:

node --version
npm --version

Recommended Method

  git clone https://github.com/lrzjason/T2ITrainer.git
  cd T2ITrainer
  setup.bat

• Handles: Virtual Environment • Dependency Installation • Model Downloads • Frontend Dependencies

The automated setup will:

1. Create a Python virtual environment
2. Install Python dependencies
3. Install Node.js dependencies for the frontend
4. Build the frontend UI
5. Download required models

Clone Repository 🌐

    git clone https://github.com/lrzjason/T2ITrainer.git
    cd T2ITrainer

Virtual Environment 🛠️

    python -m venv venv
    call venv\Scripts\activate
    pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Frontend Setup 🖥️

    cd frontend
    npm install
    npm run build
    cd ..

Backend Dependencies 📦

    pip install -r requirements.txt

Model Downloads 📥 ❗ Notice: Only download the models you want to train. Install huggingface-cli if you haven't (or update the huggingface-cli if you have an old version). You could find the download scripts in download_xxx.txt

    # NF4 Qwen Image
    hf download "lrzjason/qwen_image_nf4" --local-dir qwen_models/qwen_image_nf4/

    # NF4 Flux kontext
    hf download "lrzjason/flux-kontext-nf4" --local-dir flux_models/kontext/

    # NF4 Flux Fill for low gpu
    hf download "lrzjason/flux-fill-nf4" --local-dir flux_models/fill/

    # Kolors
    hf download Kwai-Kolors/Kolors --local-dir kolors_models/

    # SD3.5 Models
    hf download "stabilityai/stable-diffusion-3.5-large" --local-dir "sd3.5L/"

    # download original repo for lokr training
    hf download "Qwen/Qwen-Image" --local-dir qwen_models/qwen_image/
    hf download "Qwen/Qwen-Image-Edit" --local-dir qwen_models/qwen_image_edit/

For the new Node-Based Frontend UI with visualization capabilities:

Development Mode (Fastest for development):

# Terminal 1: Start backend
python backend_api.py

# Terminal 2: Start frontend (auto-reloads on changes)
cd frontend
npm run dev

Access at: http://localhost:3000

Production Mode (Optimized for performance):

# Build and serve the frontend with backend
python main.py

Access at: http://localhost:7860

Preview Mode (Pre-built optimized version):

# Terminal 1: Start backend
python backend_api.py

# Terminal 2: Serve pre-built frontend (faster than main.py)
cd frontend
npm run preview

Access at: http://localhost:7860

Performance Note: npm run dev provides the fastest experience with hot reloading, while npm run preview offers optimized performance similar to production. The python main.py approach uses npm run preview internally for better performance but still requires the backend to be running separately.

• Usage: python train_qwen_image.py --config_path config_qwen_single.json

• Usage: python train_qwen_image_edit.py --config_path config_qwen_single.json

Inpainting Model Setup

  hf download"lrzjason/qwen_image_nf4" --local-dir qwen_models/qwen_image_nf4/

For more details (example dataset):

• https://github.com/lrzjason/T2ITrainer/blob/main/doc/qwen.md

VRAM Peak

VRAM Peak

• Usage: python train_flux_lora_ui_kontext_new.py --config_path config_new_single.json

Inpainting Model Setup

  hf download"lrzjason/flux-kontext-nf4" --local-dir flux_models/kontext/

For more details (example dataset):

• https://github.com/lrzjason/T2ITrainer/blob/main/doc/image/flux_kontext.md
• https://huggingface.co/datasets/lrzjason/object_removal_alpha_kontext

Inpainting Model Setup

  hf download"lrzjason/flux-fill-nf4" --local-dir flux_models/fill/ 

For more details (example dataset):

• https://github.com/lrzjason/T2ITrainer/blob/main/doc/image/flux_fill.md
• https://huggingface.co/datasets/lrzjason/ObjectRemovalAlpha

Dev Model Installation

  hf download"black-forest-labs/FLUX.1-dev" --local-dir flux_models/dev/

• Usage (LongCat Image): python train_longcat.py --config_path config_longcat_dev.json
• Usage (LongCat Image Edit): python train_longcat_edit.py --config_path config_longcat_edit.json

LongCat Model Setup

  hf download "Meituan/LongCat-Image" --local-dir longcat_models/LongCat-Image/
  hf download "Meituan/LongCat-Image-Edit" --local-dir longcat_models/LongCat-Image-Edit/

VRAM Peak

VRAM Peak

VRAM Low

Register WandB before using it Setup WandB env

pip install wandb
wandb login

Install Tensorboard first if you choice to use Tensorboard To visualize training data, run the following command in your terminal:

tensorboard --logdir=.\logs

• Configuration Guide: 📖 CivitAI Article

• Kolors Black Image Issue: Ensure you're using FP16 Fixed VAE
• VRAM Limitations: Adjust blocks_to_swap parameter (higher values reduce memory usage)
• Windows DLL Errors: Verify VC++ Redistributable installation
• Frontend Not Loading: Ensure Node.js is installed and frontend is built (cd frontend && npm install && npm run build)
• Templates Not Found: In production builds, ensure the backend is running (python backend_api.py) before accessing the frontend
• Slow Frontend Performance: Use npm run dev for development or npm run preview for optimized local serving instead of python main.py

• https://github.com/lrzjason/T2ITrainer/blob/main/doc/change_logs.md

• 2025-07-30:
• Fix: Remove text attention mask in lora training.

• Thanks to all the contributors and sponsors for improving the project!
• Sponsor List:
• https://github.com/lrzjason/T2ITrainer/blob/main/sponsor/sponsor_list.txt

• 𝕏 Twitter: @Lrzjason
• 📧 Email: [email protected]
• 💬 QQ Group: 866612947
• 💬 WeChat ID: fkdeai
• 🎨 CivitAI: xiaozhijason

Buy me a coffee:

WeChat:

• Thanks to chenpipi0807 for Chinese translation and language switch support
• Thanks for diffusers and Terminus Research Group
• Thanks to minienglish1 and Freon in EveryDream Discord for the assistance.
• Special thanks to kohya ss for references from the training codebase.
• Thanks to Kblueleaf for coding reference on hunyuandit gradient checkpoint implementation.
• Thanks to Kolors for the open-source checkpoint.
• Thanks to comfyui for the wonderful codebase.
• Thanks to emojiiii for the setup.bat script and other updates.
• Thanks to Rohit Gandikota and related authors of Concept Sliders https://github.com/rohitgandikota/sliders