Materialist is an inverse rendering framework for material estimation and editing from single images. It leverages differentiable rendering techniques to accurately recover physically-based materials and lighting conditions from photographs.

• Single-image inverse rendering
• Material decomposition into albedo, roughness and metallic maps
• Environment map estimation
• Material editing capabilities
• Specialized rendering for transparent and translucent materials

• Mitsuba
• PyTorch

Materialist requires Python 3.10 and CUDA-compatible GPU hardware. Install dependencies with:

pip install -r requirements.txt

To reconstruct materials from a single image:

# Run the inverse rendering pipeline with default settings
./run_inverse_pipeline.sh

You can specify which example to process by entering the corresponding number when prompted.

• --img_inverse_path: Path to the input image
• --save_name: Name for saving results
• --opt_order: Optimization order for material parameters (e.g., "rm a" for roughness+metallic then albedo)
• --use_mask: Use mask during optimization, this is used for material editing purposes
• --opt_env_from: start environment map optimization from this iteration.

python inverse_img_w_mi.py --model_name=pos_mlp --opt_src=arm --opt_env_from=0 --opt_order=arm

⚠️ For real images, the following settings are recommended:

python inverse_img_w_mi.py --model_name=pos_mlp --opt_src=a --opt_env_from=2 --opt_order=rm a

if above settings do not yield good results, try optimize without using network, this will take longer time but usually yield better results:

python inverse_img_w_mi.py --model_name=none --opt_src=a --opt_env_from=2 --opt_order=rm a

# Render with default settings using example
python render_final.py --save_name="indoor" --mode="real"

# Render with transparency
python trans_edit.py --save_name="indoor" 

# Render with shadow effects using rolling envmap
python render_final.py --save_name="jinjya" --mode='rolling' --env_path='envmap/41.hdr'

• --env_path: Path to environment map (HDR)
• --save_name: Name of saved results
• --mode: Rendering mode ("real" for rendering without changes or "oi" for object insertion, "rolling" for rolling environment map)
• --input_path: Custom path for material loading
• --save_path: Custom path for saving rendered images

Results are saved to the output_imgs/{save_name}/ directory, including:

• Material maps (albedo, roughness, metallic)
• Environment maps
• Rendered images (PNG and HDR/EXR formats)
• Reconstructed mesh (.ply)

@misc{wang2025materialistphysicallybasedediting,
      title={Materialist: Physically Based Editing Using Single-Image Inverse Rendering}, 
      author={Lezhong Wang and Duc Minh Tran and Ruiqi Cui and Thomson TG and Anders Bjorholm Dahl and Siavash Arjomand Bigdeli and Jeppe Revall Frisvad and Manmohan Chandraker},
      year={2025},
      eprint={2501.03717},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2501.03717}, 
}

This project is built upon the work of many contributors. We acknowledge the use or modify of the following libraries and works: FIPT, SAM2, DepthAnything, Mitsuba. Specifically, for depth estimation, we adopted the weights from DepthAnything to better adapt to real world image inverse rendering. We also acknowledge the use of the Blender and BlenderProc for physical simulation purposes.