Du Chen^1,2* · Liyi Chen^1* · Zhengqiang Zhang^1,2 · Lei Zhang^1,2†

¹The Hong Kong Polytechnic University ²OPPO Research Institute
*Equal contribution †Corresponding author  

Comparisons with representative/SoTA ASR models (PSNR/SSIM are tested on Y channel of Ycbcr space).

We provide three versions of GSASR:

• Paper: the results we reported in our paper. (not reported) means results are not shown in our paper due to limited pages.
• Enhanced: we introduce Rotary Position Embedding (ROPE) with Flash Attention, and utilize Automatic Mixed Precision (AMP) strategy during training/inference to to reduce memory and time cost.
• Ultra Performance: based on Enhanced settings, we explore the performance upper bound of GSASR by introducing HAT-L encoder and SA1B dataset.

Toward the results of our paper, we do not use these tricks (AMP+ROPE+Flash Attention/extra training datasets) for fair comparison.

As for the pretrained models reported in our paper, please refer to Pre-trained Models (Paper Version).

• Pytorch == 2.0 (PyTorch Version must >= 2.0)
• Anaconda
• CUDA Toolkit (necessary)

Firstly, please make sure you have installed CUDA Toolkit! Since we have hand-crafted CUDA operators, you need to compile them when you run GSASR.

git clone https://github.com/ChrisDud0257/GSASR
cd GSASR
conda create --name gsasr python=3.10
conda activate gsasr
export CUDA_HOME=${path_to_CUDA} ### specify the path to cuda-11.8
pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 --index-url https://download.pytorch.org/whl/cu118
python setup_gscuda.py install # gscuda
cd TrainTestGSASR
pip install -r requirements.txt
BASICSR_EXT=True python setup_basicsr.py develop # basicsr

We have tested that the versions of CUDA from 11.0 to 12.4 are all OK.

You need to properly authenticate with Hugging Face to download our model weights. Once set up, our code will handle it automatically at your first run. You can authenticate by running

# This will prompt you to enter your Hugging Face credentials.
huggingface-cli login

You can try GSASR easily by lanching gradio demo or runing in command.

python demo_gr.py

python inference_enhenced.py \
    --input_img_path <path_to_img> \
    --save_sr_path <path_to_saved_folder> \
    --model <{EDSR_DIV2K, EDSR_DF2K, RDN_DIV2K, RDN_DF2K, SWIN_DIV2K,SWIN_DF2K, HATL_SA1B}> \
    --scale <scale> [--tile_process] [--AMP_test]

If it fails to access Huggingface, try to manually download pretrained models and specify local path with --model_path <path_to_model_weight>.

Using --tile_process and --AMP_test if memory is limited.

Please note that, in our paper, we only train GSASR on DIV2K dataset without AMP+ROPE+Flash Attention tricks for fair comparison. Due to the limited pages in our paper, we don't report the results of Swin-based model. Here, besides {EDSR, RDN}-based GSASR present in paper, we further provide the Swin-based GSASR model. The {EDSR, RDN}-based GSASR models provided bellow should exactly generate the same results as that reported in our paper (Table.1 in the main paper and Table.1 ~ Table.7 in the supplementary).

Download models from the following link.

if you have logined in the huggingface, directly execute the inference_paper.py as follows.

python inference_paper.py \
    --input_img_path <path_to_img> \
    --save_sr_path <path_to_saved_folder> \
    --model <{EDSR, RDN, SWIN}> \
    --scale <scale> [--tile_process]

To get the numerical performance in Tab.2 of the main paper. Please download cropped 720*720 size of GT images, and the corresponding LR images of DIV2K testing parts, which are utilized in our paper.

The {EDSR, RDN}-based GSASR models provided bellow should exactly generate the same PSNR/SSIM/LPIPS/DISTS results as that reported in our paper (Table.2 in the main paper and Table.8 the supplementary).

If you want to crop images all by yourself, please follow this instruction to prepare the data which could be utilized to test the computational cost.

After you download them, please test by the following command.

python inference_paper_benchmark.py \
    --input_img_path <path_to_LRx4_folder> \
    --save_sr_path <path_to_saved_folder> \
    --model <{EDSR, RDN, SWIN}> \
    --scale 4 [--tile_process]

Please indicate the "input_img_path" to your downloaded DIV2K testing parts (which is provided by us).

If you want to test GSASR on standard benchmarks with full size, please use the same commands as above. We also provide the widely-used testing benchmarks, including Set5, Set14, DIV2K-val 100, LSDIR-val 250, Urban100, Manga109, BSDS100, General100, and each GT image's corresponding LR counterparts with different scaling factors which is obtained by bicubic operation.

In inference_paper_benchmarks.py, we integrate the statistics code of test time (ms) and GPU memory (MB). In our paper, we calculate the computational cost on a single NVIDIA A100 GPU, and we input the full size image into the model, we don't use tile_process. The inference time omit the pre-processing and post-processing and record the full pipeline cost inlcuding encoder, decoder and rendering.

After inference, execute the code to estimate PSNR/SSIM/LPIPS/DISTS.

cd TrainTestGSASR/scripts/metrics/
python calculate_psnr_ssim.py --test_y_channel --gt <path_to_GT_folder> --restored <path_to_SR_folder> --scale <scale> [--suffix <suffix_of_images>]
python calculate_lpips.py  --gt <path_to_GT_folder> --restored <path_to_SR_folder> --scale <scale> [--suffix <suffix_of_images>]
python calculate_dists.py  --gt <path_to_GT_folder> --restored <path_to_SR_folder> --scale <scale> [--suffix <suffix_of_images>]

Please note that we test them on Y channel of Ycbcr space with --test_y_channel when calculating PSNR/SSIM. When calculating PSNR/SSIM/LPIPS/DISTS, we set crop_border=${scale} if the scaling factor is not larger than 8, otherwise crop_border=8.

Please follow this instruction to prepare the training and testing datasets.

Please follow this instruction to train GSASR.

Please follow this instruction to test GSASR if you further want to do it .

This project is built mainly based on the excellent BasicSR, HAT and ROPE-ViT codeframe. We appreciate it a lot for their developers.

We sincerely thank Mr.Zhengqiang Zhang for his support in the CUDA operator of rasterization.

If you find this research helpful for you, please cite our paper.

@article{chen2025generalized,
  title={Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution},
  author={Chen, Du and Chen, Liyi and Zhang, Zhengqiang and Zhang, Lei},
  journal={arXiv preprint arXiv:2501.06838},
  year={2025}
}

If you have any questions or suggestions about this project, please contact me at [email protected] .