Paper: Sha Lu, Xuecheng Xu, Dongkun Zhang, Yuxuan Wu, Haojian Lu, Xieyuanli Chen, Rong Xiong, and Yue Wang, "RING#: PR-by-PE Global Localization with Roto-translation Equivariant Gram Learning", IEEE Transactions on Robotics (TRO), 2025.
Maintainer: Sha Lu, [email protected]
- Introduction
- Key Features
- Prerequisites
- Installation
- Data Preparation
- Training
- Evaluation
- Citation
- Acknowledgments
- License
RING# is an end-to-end, PR-by-PE (Place Recognition by Pose Estimation) global localization framework that operates on BEV (Bird's Eye View) features compatible with both vision and LiDAR sensors. RING# incorporates a novel design that learns two equivariant representations from BEV features, enabling globally convergent and computationally efficient pose estimation. Comprehensive experiments on the NCLT and Oxford datasets show that RING# outperforms state-of-the-art methods in both vision and LiDAR modalities, validating the effectiveness of the proposed approach.
- End-to-End Framework: RING# is designed to be an end-to-end framework, allowing for seamless integration of all components.
- PR-by-PE Localization: Derives place recognition similarity directly from correlation-based pose estimation, reducing error accumulation compared to traditional PR-then-PE localization methods.
- Vision and LiDAR Compatibility: Supports both vision and LiDAR inputs by operating on BEV features, making it versatile for various robotic applications.
- Global Convergence: Learns equivariant representations that handle pose transformations (x, y, ΞΈ), ensuring convergence to the optimal solution without initialization.
- Computational Efficiency: Decouples pose estimation into rotation and translation, accelerated by Fast Fourier Transform (FFT) and GPU batch processing, enabling fast matching.
-
Create a virtual environment:
conda create -n ringsharp python=3.8 conda activate ringsharp
-
Clone the repository and install core dependencies:
It's highly recommended to use the exact versions specified in
requirements.txtto ensure reproducibility.git clone https://github.com/lus6-Jenny/RINGSharp.git cd RINGSharp pip install -r requirements.txt(Note: If you encounter issues, check compatibility with your CUDA and PyTorch versions.)
-
Install external dependencies from source:
Some components (fast_gicp, torch-radon) require manual compilation:
# Install fast_gicp (if needed for point cloud registration) git clone https://github.com/SMRT-AIST/fast_gicp.git --recursive cd fast_gicp python setup.py install --user # Install torch-radon (required for RING#) git clone https://github.com/matteo-ronchetti/torch-radon.git -b v2 cd torch-radon python setup.py install
-
Install the package:
# Install custom CUDA ops for glnet cd glnet/ops python setup.py develop # Install the main glnet package cd ../.. python setup.py develop
-
Download datasets:
Organize the downloaded data into directories. Example structure:
~/Data/ βββ NCLT/ β βββ yyyy-mm-dd/ (e.g., 2012-01-08) β β βββ velodyne_sync/ (Velodyne data) β β β βββ xxxxxx.bin β β β βββ ... β β βββ ground_truth/ (Ground truth data) β β β βββ groundtruth_yyyy-mm-dd.csv (e.g., groundtruth_2012-01-08.csv) β β βββ lb3/ (Ladybug camera data) β β β βββ Cam0/ β β β βββ Cam1/ β β β βββ Cam2/ β β β βββ Cam3/ β β β βββ Cam4/ β β β βββ Cam5/ β β βββ cam_param/ (Camera parameters) β β βββ U2D/ (Distortion parameters) β β βββ ... β βββ ... βββ Oxford_radar/ βββ yyyy-mm-dd-hh-mm-ss/ (e.g., 2019-01-11-13-24-51) β βββ velodyne_left/ (Left Velodyne data) β β βββ xxxxxx.bin β β βββ ... β βββ velodyne_right/ (Right Velodyne data) β β βββ xxxxxx.bin β β βββ ... β βββ gps.csv β βββ ins.csv β βββ mono_left/ β βββ mono_rear/ β βββ mono_right/ β βββ stereo/ β β βββ centre/ βββ models/ (Camera parameters) β βββ ... βββ extrinsics/ (Extrinsic parameters) β βββ ... βββ ...(Note: Adjust paths in config files according to your storage location.)
-
Preprocess images:
For image data, you need to first preprocess the images (undistort, rectify, crop, resize, etc.) and save them in the appropriate format:
# ------ NCLT Dataset ------ cd glnet/datasets/nclt python image_preprocess.py # The generated images will be saved in the 'lb3_u_s_384/Cam{0,1,2,3,4,5}' directories in jpg format # ------ Oxford Radar Dataset ------ cd glnet/datasets/oxford python image_preprocess.py # The generated images will be saved in the 'mono_left_rect', 'mono_rear_rect', 'mono_right_rect', 'stereo/centre_rect' directories in png format
-
Generate training and evaluation data:
These scripts create
.picklefiles containing metadata, poses, and other information for training and evaluation:# ------ NCLT Dataset ------ cd glnet/datasets/nclt # Generate training and validation pickles python generate_training_tuples.py --dataset_root ~/Data/NCLT # Generate evaluation pickle python generate_evaluation_sets.py --dataset_root ~/Data/NCLT # ------ Oxford Radar Dataset ------ cd glnet/datasets/oxford # Generate training and validation pickles python generate_training_tuples.py --dataset_root ~/Data/Oxford_radar # Generate evaluation pickle python generate_evaluation_sets.py --dataset_root ~/Data/Oxford_radar # ------ Optional Data Generation ------ # Generate bev images for LiDAR data (use --bev flag) python generate_training_tuples.py --dataset_root ~/Data/NCLT --bev # Generate panorama images for vision data (use --sph flag) python generate_training_tuples.py --dataset_root ~/Data/NCLT --sph # defaule scale of panorama size is 2
(Note: The generated
.picklefiles will be saved in thedataset_rootdirectory.) -
Configure the dataset and training parameters:
The configuration files are located in the
glnet/configdirectory. You can use the provided example files as templates:config_nclt.txt: Configuration for the NCLT dataset.config_oxford.txt: Configuration for the Oxford Radar dataset.ring_sharp_v_nclt.txt: Configuration for the RING#-V model for the NCLT dataset.netvlad_pretrain_oxford.txt: Configuration for the NetVLAD model for the Oxford Radar dataset.- ...
You can modify these files to suit your specific dataset and training parameters. For example, you can set the dataset type, root directory, evaluation set, loss function, batch size, learning rate, and other parameters.
The training script is located in the tools directory. You can run the training script with the following command:
cd tools
python train.py --dataset_type nclt \
--dataset_root ~/Data/NCLT \
--exp_name ring_sharp_v_nclt_run1 \
--config ../glnet/config/config_nclt.txt \
--model_config ../glnet/config/ring_sharp_v_nclt.txt \
# --resume \
# --weight xxx.pthKey flags:
--dataset_type: Specify the dataset type (e.g., nclt, oxford).--dataset_root: Path to the dataset root directory.--exp_name: Name of the experiment (used for saving checkpoints).--config: Path to the configuration file.--model_config: Path to the model configuration file.--resume: Flag to resume training from a checkpoint.--weight: Path to the pre-trained model weights.
-
Evaluate the model of RING# (RING#-V, RING#-L):
- Pose estimation evaluation:
cd tools python evaluate_ours_pe.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name ring_sharp_v_nclt_run1 \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/ring_sharp_v.txt \ --weight xxx.pth
- Place recognition and global localization evaluation:
cd tools python evaluate_ours_gl.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name ring_sharp_v_nclt_run1 \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/ring_sharp_v_nclt.txt \ --weight xxx.pth \ --n_rerank 1000 \ --refine \ # --one_shot (for one-stage global localization)
- Pose estimation evaluation:
-
Evaluate the model of visual baselines:
- Pose estimation evaluation (SIFT + NN, SuperPoint + NN, SuperPoint + SuperGlue):
cd glnet/models/extractor_matcher python evaluate_pe.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name netvlad_nclt_run1 \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../../config/netvlad_pretrain_nclt.txt \ --extractor superpoint \ --matcher superglue \ --pnp
- Place recognition global localization evaluation (NetVLAD + SS, Patch-NetVLAD + SS, SFRS + SS, AnyLoc + SS, vDiSCO + SS, SS: SuperPoint + SuperGlue):
cd glnet/models/extractor_matcher python evaluate_gl.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name netvlad_nclt_run1 \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../../config/netvlad_pretrain_nclt.txt \ --weight xxx.pth \ --extractor superpoint \ --matcher superglue \ --pnp \ # one_shot (for one-stage global localization)
- Pose estimation evaluation (SIFT + NN, SuperPoint + NN, SuperPoint + SuperGlue):
-
Evaluate the model of LiDAR-based baselines:
- Pose estimation evaluation:
cd tools # RING, RING++ python evaluate_ring_pe.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name ring_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/ring_nclt.txt \ --icp_refine # EgoNN python evaluate_egonn_pe.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name egonn_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/egonn_nclt.txt \ --icp_refine # LCDNet python evaluate_lcdnet_pe.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name lcdnet_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/lcdnet_nclt.txt \ --icp_refine # DiSCO, OverlapTransformer python evaluate_others_pe.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name disco_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/disco_nclt.txt \ --icp_refine
- Place recognition and global localization evaluation:
cd tools # RING, RING++ python evaluate_ours_gl.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name ring_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/ring_nclt.txt \ --icp_refine \ # --one_shot (for one-stage global localization) # EgoNN python evaluate_egonn_gl.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name egonn_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/egonn_nclt.txt \ --icp_refine # --one_shot (for one-stage global localization) # LCDNet python evaluate_lcdnet_gl.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name lcdnet_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/lcdnet_nclt.txt \ --icp_refine # --one_shot (for one-stage global localization) # DiSCO, OverlapTransformer python evaluate_others_gl.py --dataset_type nclt \ --dataset_root ~/Data/NCLT \ --exp_name disco_nclt \ --eval_set test_2012-02-04_2012-03-17_20.0_5.0.pickle \ --model_config ../glnet/config/disco_nclt.txt \ --icp_refine # --one_shot (for one-stage global localization)
- Pose estimation evaluation:
Key flags:
--dataset_type: Specify the dataset type (e.g., nclt, oxford).--dataset_root: Path to the dataset root directory.--exp_name: Name of the experiment (used for saving checkpoints).--eval_set: Path to the evaluation set file.--model_config: Path to the model configuration file.--weight: Path to the trained model weights.--extractor: Feature extractor type (e.g., superpoint).--matcher: Feature matcher type (e.g., superglue).--pnp: Flag to use PnP (Perspective-n-Point) for pose estimation.--icp_refine: Flag to use ICP (Iterative Closest Point) for refining the pose estimation.--one_shot: Flag for one-stage global localization.--n_rerank: Number of nearest neighbors for reranking.--refine: Flag to use 3-DoF pose refinement for global localization with RING#.--radius: List of revisit thresholds for evaluation.--viz: Flag to visualize the evaluation results.
If you find this work useful, please consider citing:
@ARTICLE{10891747,
author={Lu, Sha and Xu, Xuecheng and Zhang, Dongkun and Wu, Yuxuan and Lu, Haojian and Chen, Xieyuanli and Xiong, Rong and Wang, Yue},
journal={IEEE Transactions on Robotics},
title={RING#: PR-By-PE Global Localization With Roto-Translation Equivariant Gram Learning},
year={2025},
volume={41},
pages={1861-1881},
doi={10.1109/TRO.2025.3543267}}We thank the developers of the following open-source projects for their contributions:
The code is released under the MIT License.