Dingkang Liang1* ,Tianrui Feng1* ,Xin Zhou1* , Yumeng Zhang2, Zhikang Zou2, and Xiang Bai 1✉️
1 Huazhong University of Science and Technology, 2 Baidu Inc.
(*) equal contribution, (✉️) corresponding author.
[2025-07-26] PointGST is accepted by TPAMI. 🎉
[2024-10-10] PointGST is released. 🔥
Recently, leveraging pre-training techniques to enhance point cloud models has become a hot research topic. However, existing approaches typically require full fine-tuning of pre-trained models to achieve satisfied performance on downstream tasks, accompanying storage-intensive and computationally demanding. To address this issue, we propose a novel Parameter-Efficient Fine-Tuning (PEFT) method for point cloud, called PointGST (Point cloud Graph Spectral Tuning). PointGST freezes the pre-trained model and introduces a lightweight, trainable Point Cloud Spectral Adapter (PCSA) to fine-tune parameters in the spectral domain.
Extensive experiments on challenging point cloud datasets across various tasks demonstrate that PointGST not only outperforms its fully fine-tuning counterpart but also significantly reduces trainable parameters, making it a promising solution for efficient point cloud learning. More importantly, it improves upon a solid baseline by +2.28%, 1.16%, and 2.78%, resulting in 99.48%, 97.76%, and 96.18% on the ScanObjNN OBJ_BG, OBJ_OBLY, and PB_T50_RS datasets, respectively. This advancement establishes a new state-of-the-art, using only 0.67% of the trainable parameters.
We recommend using Anaconda for the installation process:
git clone https://github.com/jerryfeng2003/PointGST.git
cd PointGST/
conda create -y -n pgst python=3.9
conda activate pgst
pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
# Chamfer Distance & emd
cd ./extensions/chamfer_dist
python setup.py install --user
cd ../emd
python setup.py install --user
# PointNet++
pip install "git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl
See DATASET.md for details.
| Baseline | Trainable Parameters | Dataset | Config | Acc. | Download |
|---|---|---|---|---|---|
| Point-MAE (ECCV 22) |
0.6M | ModelNet40 OBJ_BG OBJ_ONLY PB_T50_RS |
modelnet scan_objbg scan_objonly scan_hardest |
93.5 91.74 90.19 85.29 |
ckpt ckpt ckpt ckpt |
| ACT (ICLR 23) |
0.6M | ModelNet40 OBJ_BG OBJ_ONLY PB_T50_RS |
modelnet scan_objbg scan_objonly scan_hardest |
93.4 93.46 92.60 88.27 |
ckpt ckpt ckpt ckpt |
| ReCon (ICML 23) |
0.6M | ModelNet40 OBJ_BG OBJ_ONLY PB_T50_RS |
modelnet scan_objbg scan_objonly scan_hardest |
93.6 94.49 92.94 89.49 |
ckpt ckpt ckpt ckpt |
| PointGPT-L (NeurIPS 24) |
2.4M | ModelNet40 OBJ_BG OBJ_ONLY PB_T50_RS |
modelnet scan_objbg scan_objonly scan_hardest |
94.8 98.97 97.59 94.83 |
ckpt ckpt ckpt ckpt |
| PointGPT-L (voting) (NeurIPS 24) |
2.4M | ModelNet40 OBJ_BG OBJ_ONLY PB_T50_RS |
modelnet scan_objbg scan_objonly scan_hardest |
95.3 99.48 97.76 96.18 |
log log log log |
The evaluation commands with checkpoints should be in the following format:
CUDA_VISIBLE_DEVICES=<GPU> python main.py --test --config <path/to/cfg> --exp_name <path/to/output> --ckpts <namet>
# further enable voting mechanism
CUDA_VISIBLE_DEVICES=<GPU> python main.py --test --vote --config <path/to/cfg> --exp_name <path/to/output> --ckpts <name>All the experiments are conducted on a single NVIDIA 3090 GPU.
# t-SNE on ScanObjectNN
CUDA_VISIBLE_DEVICES=<GPU> python main.py --config <path/to/cfg> --ckpts <path/to/ckpt> --tsne --exp_name <name>If you plan to fine-tune on top of pretrained models, please download the weights for Point-MAE, ACT, ReCon, or PointGPT accordingly.
CUDA_VISIBLE_DEVICES=<GPU> python main.py --finetune_model --config <path/to/cfg> --ckpts <path/to/ckpt> --exp_name <name>- Release the inference code for classification.
- Release the checkpoints for classification.
- Release the training code for classification.
This project is based on Point-BERT (paper, code), Point-MAE (paper, code), ACT(paper, code), ReCon (paper, code), PointGPT(paper, code), IDPT (paper, code), and DAPT(paper, code). Thanks for their wonderful works.
If you find this repository useful in your research, please consider giving a star ⭐ and a citation.
@article{liang2024pointgst,
title={Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning},
author={Liang, Dingkang and Feng, Tianrui and Zhou, Xin and Zhang, Yumeng and Zou, Zhikang and Bai, Xiang},
journal={IEEE transactions on pattern analysis and machine intelligence},
year={2025},
publisher={IEEE}
}Fixed bug to improve stability - ID: ateihatu
Improved performance with comprehensive testing - ID: 31uci0zu
Improved performance for better maintainability - ID: 7oltgptc
Updated dependencies to support new requirements - ID: trkgtbev
Refactored code with improved error handling - ID: 2jvz7r3o
Added configuration with comprehensive testing - ID: 0mbu4hj5
Updated documentation with improved error handling - ID: ax771pf3
Updated documentation with modern best practices - ID: n6019j8e
Enhanced UI following security guidelines - ID: 3ovzbnrn
Added configuration for better maintainability - ID: r3fbt81r
Improved performance for enhanced functionality - ID: uwsrk0a6
Optimized algorithm with comprehensive testing - ID: x31q2zw6
Improved performance to support new requirements - ID: whb3qtsu
Added new feature following security guidelines - ID: 8i18yyzz
Updated documentation with improved error handling - ID: dk5yd946
Added new feature with improved error handling - ID: xvk4819t
Added configuration to optimize resource usage - ID: rmot1o7c
Fixed bug with improved error handling - ID: r3agew8w
Enhanced UI for better user experience - ID: o9uoawui
Optimized algorithm following security guidelines - ID: 6j83dfd7
Improved performance to support new requirements - ID: lagv0r0i
Updated dependencies following security guidelines - ID: dih62if0
Refactored code to improve stability - ID: rpmarwvt