OGGSplat: Open Gaussian Growing for Generalizable Reconstruction with Expanded Field-of-View

Paper | Project Page | Code

Official codebase for OGGSplat, presented in the paper: OGGSplat: Open Gaussian Growing for Generalizable Reconstruction with Expanded Field-of-View

Created by Yanbo Wang*, Ziyi Wang*, Wenzhao Zheng, Jie Zhou, Jiwen Lu†.

🧠 Overview

OGGSplat is a novel generalizable 3D reconstruction framework that extends the field-of-view from sparse input images by leveraging semantic priors. It introduces an Open Gaussian Growing approach that combines RGB-semantic joint inpainting guided by bidirectional diffusion models to extrapolate unseen regions beyond the input views. The inpainted images are then lifted back into 3D space for progressive optimization of Gaussian parameters, enabling efficient and high-quality semantic-aware scene reconstruction.

✅ TO DO List

• Release Checkpoints
  Publicly release all pretrained model checkpoints.

• Inference Code
  Provide inference scripts to perform scene extrapolation from sparse input views (scannet++) using pretrained models.

• Evaluation Benchmark
  Release the Gaussian Outpainting (GO) benchmark suite, including quantitative metrics for semantic consistency and visual plausibility.

• Demo Code
  Provide demo scripts to perform scene extrapolation from any 2 input views.

• Training Code
  Provide the full training pipeline, covering Gaussian initialization, diffusion-based inpainting, and progressive optimization procedures.

📦 Installation

• Please refer to installation.md for detailed instructions on setting up the environment and installing all required dependencies for training and evaluation.
• To enable training and inference, pre-trained APE text/image encoder weights are required. For convenience, we provide the pre-trained weights here: 📥 APE Weights.

📚 Datasets

Our training and inference are based on the ScanNet++ dataset. We download the raw data from the official ScanNet++ homepage and preprocess it using a customized version of the ScanNet++ toolkit provided by SplaTAM.

📂 Model Checkpoints

Model Name	Function Description	Download Link
Autoencoder	Encode or decode APE features to adjust their dimensions	📥 Google Drive
Gaussian Init.	Initialize Gaussians with semantic-aware feature embeddings	📥 Google Drive
RGB UNet	Denoising module for RGB image inpainting tasks	📥 Google Drive
Sem. VAE	Variational autoencoder for semantic feature representation	📥 Google Drive
Sem. UNet	Denoising module for semantic map inpainting tasks	📥 Google Drive
ControlNet	Control module to guide semantic generation using RGB conditions	📥 Google Drive

🚀 Inference for Scannet++

This section provides instructions to perform Gaussian growing inference on the ScanNet++ dataset using our pre-trained model. Before proceeding, please ensure that you have downloaded the dataset and the corresponding checkpoint files.

You need to specify the dataset path and checkpoint locations in the configuration file configs/oggsplat_infer.yml. Additionally, you can customize the input by editing the select_frames parameter to choose any two context views as input.

Once the configuration is set, run the following command to start the Gaussian growing process:

python gaussian_growing.py --seed <seed_value> --config <path_to_config_file>

📄 Citation

If you find our work helpful, please consider citing:

@article{wang2025oggsplat,
  title={OGGSplat: Open Gaussian Growing for Generalizable Reconstruction with Expanded Field-of-View},
  author={Wang, Yanbo and Wang, Ziyi and Zheng, Wenzhao and Zhou, Jie and Lu, Jiwen},
  journal={arXiv preprint arXiv:2506.05204},
  year={2025}
}