English | 简体中文

ComfyUI custom nodes plugin based on SDMatte for interactive image matting.

📺 Video Tutorial: ComfyUI-SDMatte Tutorial
🔧 Example Workflow: Superior Image Cropping and Mask Refinement Workflow
💡 Recommended: Watch the video tutorial first to understand the usage, then download the workflow for practice

SDMatte is an interactive image matting method based on Stable Diffusion, developed by the vivo Camera Research team and accepted by ICCV 2025. This method leverages the powerful priors of pre-trained diffusion models and supports multiple visual prompts (points, boxes, masks) for accurately extracting target objects from natural images.

This plugin integrates SDMatte into ComfyUI, providing a simple and easy-to-use node interface focused on trimap-guided matting functionality with built-in VRAM optimization strategies.

Original Image	Trimap	Matting Result
		Alpha mask output

Example workflow demonstrating SDMatte's high-precision matting capabilities with trimap guidance.

• 🎯 High-Precision Matting: Based on powerful diffusion model priors, capable of handling complex edge details
• 🖼️ Trimap Guidance: Supports trimap-guided precise matting
• 🚀 VRAM Optimization: Built-in mixed precision, attention slicing, and other memory optimization strategies
• 🔧 ComfyUI Integration: Fully compatible with ComfyUI workflow system
• 📥 Automatic Model Download: Automatically downloads model weights on first use
• 📱 Flexible Sizes: Supports multiple inference resolutions (512-1024px)

Place this plugin in the ComfyUI custom nodes directory:

cd ComfyUI/custom_nodes/
git clone https://github.com/flybirdxx/ComfyUI-SDMatte.git

ComfyUI will automatically install the dependencies in requirements.txt on startup:

• diffusers
• timm
• einops
• lazyconfig
• safetensors

The plugin automatically downloads required config files, no manual action needed.

SDMatte is built on top of Stable Diffusion 2.1. On first use, it automatically downloads necessary config files from Hugging Face (no model weights):

• Auto-downloads to: ComfyUI/models/diffusers/stable-diffusion-2-1-base/
• Downloads only config files: config.json, tokenizer files, etc.
• No large weight files downloaded, saving disk space

If you need to manually download the complete Stable Diffusion 2.1 model:

# Navigate to diffusers directory
cd ComfyUI/models/diffusers/

# Download Stable Diffusion 2.1 base model
# Method 1: Using huggingface_hub
pip install huggingface_hub
huggingface-cli download stabilityai/stable-diffusion-2-1 --local-dir stable-diffusion-2-1-base

# Method 2: Using git lfs (if installed)
git lfs install
git clone https://huggingface.co/stabilityai/stable-diffusion-2-1 stable-diffusion-2-1-base

SDMatte model weights will be downloaded automatically.

The first time you use the Apply SDMatte node, it will automatically check for and download the necessary SDMatte model weights from Hugging Face. The models will be stored in: ComfyUI/models/SDMatte/

You can select between the standard (SDMatte.safetensors) and enhanced (SDMatte_plus.safetensors) versions directly within the node.

Restart ComfyUI to load the new custom nodes.

• Function: Loads the model and applies it for matting in a single node.
• Input:
  • ckpt_name: Select the model to use (SDMatte.safetensors or SDMatte_plus.safetensors). It will be downloaded automatically if not found.
  • image: Input image (ComfyUI IMAGE format)
  • trimap: Trimap mask (ComfyUI MASK format)
  • inference_size: Inference resolution (512/640/768/896/1024)
  • is_transparent: Whether the image contains transparent areas
  • output_mode: Output mode (alpha_only, matted_rgba, matted_rgb)
  • mask_refine: Enable mask refinement to reduce background interference
  • trimap_constraint: Strength of the trimap constraint for refinement
  • force_cpu: Force CPU inference (optional)
• Output:
  • alpha_mask: Alpha mask of the matting result
  • matted_image: The matted image result

1. Load Image: Load the image that needs matting
2. Create Trimap: Use drawing tools or other nodes to create trimap
   • Black (0): Definite background
   • White (1): Definite foreground
   • Gray (0.5): Unknown region
3. Apply SDMatte: Apply matting
4. Preview Image: Preview matting result

• Inference Resolution: 1024 (highest quality) or 768 (balanced performance)
• Transparent Flag: Set according to whether input image has transparent channel
• Force CPU: Use only when GPU VRAM is insufficient

• Input Image: Automatically resized to inference resolution, normalized to [-1, 1]
• Trimap: Resized to inference resolution, mapped to [-1, 1] range
• Output: Resized back to original resolution, clamped to [0, 1] range

The plugin has built-in memory optimization strategies (automatically enabled):

• Mixed Precision: Uses FP16 autocast to reduce VRAM usage
• Attention Slicing: SlicedAttnProcessor(slice_size=1) maximizes VRAM savings
• Memory Cleanup: Automatically clears CUDA cache before and after inference
• Device Management: Smart device allocation and model movement

• Weight Formats: Supports .pth and .safetensors formats
• Safe Loading: Handles omegaconf objects, supports weights_only mode
• Nested Structure: Automatically handles complex checkpoint structures
• Error Recovery: Multiple fallback mechanisms ensure successful loading

A: Ensure the plugin directory structure is correct, restart ComfyUI, check console for error messages.

A: Check SDMatte.safetensors file path, ensure base model directory structure is complete, view console for detailed error messages.

A: Try reducing inference resolution, enable force_cpu option, or close other VRAM-consuming programs.

A: Optimize trimap quality, ensure accurate foreground/background/unknown region annotations, try different inference resolutions.

A: First run needs to compile CUDA kernels, subsequent inference will be significantly faster.

A:

• SDMatte.safetensors (Standard): Smaller file (~11GB), faster inference, suitable for most scenarios
• SDMatte_plus.safetensors (Enhanced): Larger file, higher accuracy, suitable for professional use with extremely high quality requirements
• Recommend testing with standard version first, upgrade to enhanced version if higher quality is needed

• ComfyUI: Latest version
• Python: 3.8+
• PyTorch: 1.12+ (CUDA support recommended)
• VRAM: 8GB+ recommended (CPU inference supported)
• Base Model: Stable Diffusion 2.1 base (located in ComfyUI/models/diffusers/stable-diffusion-2-1-base/)
• Dependencies: diffusers, timm, einops, lazyconfig, safetensors

• 🔧 Architecture Optimization:
  • Modified to directly use Stable Diffusion 2.1 model from global diffusers directory
  • Removed dependency on local stable-diffusion-2.1 directory
  • Reduced disk space usage by avoiding duplicate model downloads
• 📚 Documentation Updates:
  • Updated installation instructions with base model download steps
  • Updated system requirements to clarify base model dependency

• 🔄 Model Format Update:
  • Migrated from .pth to .safetensors format for better security and performance
  • Updated model download URLs to use Hugging Face repository (1038lab/SDMatte)
  • Improved model loading with SafeTensors library for safer weight handling
• 🔧 Technical Improvements:
  • Enhanced model loading stability with better error handling
  • Optimized memory usage during model loading process
  • Improved compatibility with latest ComfyUI versions
• 📚 Documentation Updates:
  • Updated installation instructions to reflect new model format
  • Added information about SafeTensors format benefits

• ✨ New Features:
  • Implemented automatic model downloading and checking. Models are now stored in ComfyUI/models/SDMatte/.
• 🔧 Improvements:
  • Merged SDMatte Model Loader and SDMatte Apply nodes into a single Apply SDMatte node for a more streamlined workflow.
  • Refactored code for better stability.

• ✨ New Features:
  • Added image output alongside alpha mask output
  • Support for transparent background matting mode
  • Added multiple output modes: alpha_only, matted_rgba, matted_rgb
  • Added mask refinement feature using trimap constraints to filter unwanted regions
  • Added trimap_constraint parameter to control constraint strength
  • Added detailed tooltips for all parameters
• 🔧 Improvements:
  • Improved alpha mask processing logic to reduce background interference
  • Optimized foreground region extraction algorithm
  • Enhanced low-confidence region filtering mechanism
• 📚 Documentation:
  • Added example workflow links
  • Added video tutorial links
  • Improved usage instructions and parameter explanations
• 🔧 Improvements:
  • Improved VRAM optimization strategies
  • Enhanced model loading stability
  • Optimized inference performance

• 🎉 Initial Release:
  • Basic SDMatte model integration
  • Support for trimap-guided matting
  • Built-in VRAM optimization features
  • Support for multiple inference resolutions

• Example Workflow: Superior Image Cropping and Mask Refinement Workflow
• Video Tutorial: ComfyUI-SDMatte Tutorial
• Original Paper: SDMatte: Grafting Diffusion Models for Interactive Matting (ICCV 2025)
• Original Code: vivoCameraResearch/SDMatte
• Model Weights: LongfeiHuang/SDMatte

This project follows the MIT license. The original SDMatte project also uses the MIT license.

Thanks to the vivo Camera Research team for developing the excellent SDMatte model, and to the Stable Diffusion and ComfyUI communities for their contributions.

If you have any questions or suggestions, please submit an Issue on GitHub.

Note: This plugin is a third-party implementation and is not directly affiliated with the original SDMatte team. Please ensure compliance with relevant license terms before use.