PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-forward Planar Splatting

Changkun Liu^1,2*, Bin Tan²*, Zeran Ke^2,3, Shangzhan Zhang^2,4, Jiachen Liu⁵, Ming Qian^2,3,
Nan Xue^2†, Yujun Shen², Tristan Braud¹

*Equal Contribution; ^†Corresponding author

@inproceedings{
liu2025plana3r,
title={PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-forward Planar Splatting},
author={Changkun Liu and Bin Tan and Zeran Ke and Shangzhan Zhang and Jiachen Liu and Ming Qian and Nan Xue and Yujun Shen and Tristan Braud},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=YTwRZP8mNO}
}

• [Oct 22, 2025] An improved new model, PLANA3Rv2, which outputs 3D planes without requiring image camera intrinsics, has been released and achieves improved results.
• [Oct 22, 2025] Code and pre-trained model is available.

PLANA3R is an end-to-end transformer-based model for two-view metric 3D reconstruction and metric relative pose estimation, specifically designed for structured indoor scenes. It represents scenes using sparse 3D planar primitives.

We build upon planar primitives introduced in PlanarSplatting, and leverage its CUDA-based differentiable renderer for supervision. Therefore, even if what it directly outputs is sparse planar primitives, it still supports the dense depth/normal rendering.

# cuda 12.1, python 3.8, torch 2.4.1

conda create -n plana3r python=3.8

# install pytorch
pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu121

# install pytorch3d
pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable"

# install deps
pip install -r requirements.txt

# install diff_rect_rasterization
pip install planar_splatting/submodules/diff-rect-rasterization

We provide several pre-trained models:

The PLANA3Rv2.pth model incorporates two additional heads, leading to performance improvements as detailed in Section: Features of Improved Model.

You can run PLANA3R with an interactive demo for your own data as follows:

python demo.py

Referring to VGGT and DUSt3R, we have added points map head and FoV head to the model. During training, these heads can be supervised using loss functions similar to those employed in DUSt3R and VGGT, allowing the model to handle uncalibrated and unposed images directly.

The codes for PLANA3R and PLANA3Rv2 are respectively located at plana3r/model_plana3r_v1.py and plana3r/model_plana3r_v2.py. The table below shows performance improvements using the same data and hyperparameters.

The best results are in bold.

• Release the training code

This work was supported by Ant Group Research Intern Program and Ant Group Postdoctoral Program. This project is developed based on several fantastic repos:

• DUSt3R
• MASt3R
• PlanarSplatting
• VGGT
• Nope-SAC
• 3DGS
• Metric3Dv2

We are extremely grateful for the contributions of these projects and their communities, whose hard work has greatly propelled the development of the field and enabled our work to be realized.