Project Page | Paper | Data

• [5/16/2025] 3D gaussian segmentation viewer
• [5/2/2025] Thx Tomi for releasing code for wheat head morphology calculation at wheatheadsmorphology!
• [4/30/2025] Initial code release

• Share Docker setup
• Script to to get initial masks
• Data to ETH research collection including laser scans

The setup should be very similar to the original 3D Gaussian Splatting except we used a modified version of differential gaussian rasterization with support of depth & alpha rendering, and an additional flashsplat-rasterization submodule.

git clone --recursive https://github.com/zdwww/Wheat-3DGS.git
conda env create --file environment.yml
conda activate wheat3dgs

Data used for Wheat3DGS and the baseline results for each plot (from 461 to 467) are structured as follows:

plot_46x/
├── images/ # 36 images captured by FIP
│   ├──FPWW036_SR0461_FIP2_cam_01.png 
│   └──FPWW036_SR0461...
├── sparse/ # camera intrinsics/extrinsics in COLMAP format
├── yolosam.zip # please unzip to extract 2D masks formatted for Wheat3DGS
│   ├──bboxes/ # YOLO bounding boxes per image (xyxy)
│   │  ├──FPWW036_SR0461_1_FIP2_cam_01.pt
│   │  └──...
│   └──masks/ # 2D segmention mask corresponds to each box (per instance per image) 
│   │  ├──FPWW036_SR0461_1_FIP2_cam_01_000.png
│   │  ├──FPWW036_SR0461_1_FIP2_cam_01_001.png
│   │  └──...
│   ├──yolo_vis/ # for visualization purpose only
│   └──sam_vis/ # for vis only
├── semantics/ # combined 2D masks per image for FruitNeRF baseline
├── manual_label/ # contains a randomly selected test view image with human-annotated wheat heads
└── transforms.json # data in NeRF format

Please unzip yolosam.zip for each plot before running the pipeline. The image names are structured as follows. For example, in FPWW036_SR0461_FIP2_cam_01.png: FPWW036_SR0461 is shared by all images from the same plot; FIP2 indicates the batch ID, which may appear as 1_FIP2, 6_FIP, or FIP2; and the final element 01 is the camera ID, ranging from 01 to 12. Our data preprocessing functions already handle the naming convention correctly. To get a clearer view of how the cameras are positioned in 3D space and how the train/test views are distributed, please run vis_cam.py for visualization.

The majority of the Wheat3DGS pipeline can be executed by running this script.

sbatch run_wheat_3dgs.sh

Step 1. To train a vanilla 3DGS scene reconstruction using wheat plot images as initialization:

python train_vanilla_3dgs.py -s <dataset_path> -m <model_path> --resolution 1 --eval

Note that resolution is set to 1 to prevent image downscaling, and the training/test split for evaluation follows the default setting of our dataset.

You can also run render.py and metrics.py with the same input arguments to evaluate the quality of novel view synthesis on the reconstructed wheat fields (see Sec. 5.1).

Step 2 (Optional). Although wheat head bounding boxes generated by YOLOv5 and 2D segmentations by SAM are already provided in our dataset, please refer to this Colab notebook to reproduce results.

Step 3. To run 3D segmentation on a wheat plot to identify and associate decoupled 2D wheat heads:

python run_3d_seg.py -s <dataset_path> -m <model_path> --iou_threshold 0.6 --exp_name run1

Step 4. To generate a video rotating around the reconstructed wheat field for qualitative evaluation of the 3D segmentation results (as shown on our project page), run:

python render_360.py -s <dataset_path> -m <model_path> --render_type <field or head> --exp_name run1 --n_frames 200 --framerate 20 --elevation 45

Results will be saved as a video <model_path>/wheat_head/<exp_name>/wheat_field_360.mp4 for the whole wheat field and a list of videos under the directory .../<exp_name>/wheat_head_360 for each wheat head id.

Step 5. To quantitatively evaluate the quality of 3D segmentation and reproduce Table 2 in the paper, project the 3D segmentation into 2D by

python eval_wheatgs.py -s <dataset_path> -m <model_path> --exp_name run1 --skip_train --load_counts

Rendered 2D segmentation will be saved under model_path/test/ours_<iteration>/segmentation and overlay, where they can be used for metric calculation.

To reproduce the baseline results presented in the paper (i.e. FruitNeRF), please refer to the original repository and the scripts in the scripts folder.

Our viewer is based on the implementation of viser, allowing users to freely explore the reconstructed wheat field with overlaid 3D segmentation of wheat heads. Please refer to the scripts under folder wheat3dgsviewer for usage instructions.

Our implementation is based on the original 3D Gaussian Splatting and FlashSplat. We thank the authors for their revolutionary work and open-source contributions.

This project is licensed under the MIT License - see the LICENSE file for details. Please note that the 3D Gaussian Splatting code base, however, is licensed under a different license, which can be found in the LICENSE_3DGS file.

If you find our paper useful, please cite us:

@InProceedings{Zhang_Gajardo_2025_CVPR,
    author    = {Zhang, Daiwei and Gajardo, Joaquin and Medic, Tomislav and Katircioglu, Isinsu and Boss, Mike and Kirchgessner, Norbert and Walter, Achim and Roth, Lukas},
    title     = {Wheat{3DGS}: {I}n-field {3D} {R}econstruction, {I}nstance {S}egmentation and {P}henotyping of {W}heat {H}eads with {G}aussian {S}platting},
    booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops},
    month     = {June},
    year      = {2025},
    pages     = {5369-5379}
}