Xin Wang¹, Kai Chen¹, Jiaming Zhang², Jingjing Chen¹, Xingun Ma^1
1Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University
²Hong Kong University of Science and Technology

To set up the required environment, please follow the installation instructions provided in the Train-Time APT repository.

Before training or evaluating the models, you'll need to prepare the necessary datasets. Detailed instructions on downloading, preprocessing, and organizing the data can be found in DATASETS.md.

ADV_DIR can be generated using the Train-Time APT repository.

This project provides scripts for test-time tuning and evaluating various prompt designs. You can find all scripts in the ./scripts directory.

Here are examples of how to train and evaluate different Test-Time Adversarial Prompt Tuning using a ViT-B/16 backbone in a zero-shot setting:

• TAPT-VLI (Test-Time Adversarial V-L Independent Prompt):

  ./scripts/vlip/TAPT_VLI_0shots_step1_eps1.sh

• TAPT-VLJ (Test-Time Adversarial V-L Joint Prompt):

  ./scripts/vljp/TAPT_VLJ_0shots_step1_eps1.sh
  

This repository is built upon PromptAlign and TPT. Thanks for those well-organized codebases.

@inproceedings{wang2025tapt,
  title={TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models},
  author={Wang, Xin and Chen, Kai and Zhang, Jiaming and Chen, Jingjing and Ma, Xingjun},
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
  pages={19910--19920},
  year={2025}
}