This repository contains the source code for the paper titled PViT: Prior-augmented Vision Transformer for Out-of-distribution Detection.

Note: We apologize for any inconvenience caused by the current state of the code. Due to ongoing improvements, the training and testing scripts are located in separate folders. We are working on consolidating the code into a single repository, which will be released upon the acceptance of the paper.

Install the ./requirements.txt.

Then install the required packages by

chmod +x install_packages.sh
./install_packages.sh

To set up the dataset folder structures, please refer to the README.md file located in the ./dataloaders directory.

Download ILSVRC2012_img_train.tar and ILSVRC2012_img_val.tar from the official ImageNet website. And use ./dataloaders/assets/extract_ILSVRC.sh to unzip the zip files.

To download iNaturalist, SUN, and Places

wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/SUN.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/Places.tar.gz

Download Textures from the official website. Download OpenImage-O from the official website.

For other datasets, please refer to the OpenOOD

Download resnet50-supcon.pt from the link and put it in the directory pretrained_models as ./pretrained_models/resnet50-supcon.py.

Other prior models will be automatically downloaded from Huggingface or Pytorch.

To train To train PViT on IMAGENET-1k, please run

python main.py --model_name pvit --id_data_name imagenet1k --ood_data_name inaturalist --ood_detectors pvit --pvit --batch_size 256 --num_workers 1 --prior_model vit-b-16 --score cross_entropy --seed 0 --train

To train the PViT model with different prior models, modify the --prior_model argument. The available options are:

• vit_imagenet for Google ViT
• vit-b-16 for DeiT
• vit-lp for ViT-LP
• resnet50-supcon for ResNet50-SupCon
• regnet-y-16gf-swag-e2e-v1 for RegNet
• vit-b16-swag-e2e-v1 for ViT-Swag

To train PViT on CIFAR100, please run

python main.py --model_name pvit --id_data_name cifar100 --ood_data_name cifar10 --ood_detectors pvit --pvit --batch_size 512 --num_workers 1 --prior_model vit_cifar100 --score cross_entropy --seed 0 --train

To run experiments, run

python main.py --model_name pvit --id_data_name imagenet1k --ood_data_name inaturalist --ood_detectors pvit --batch_size 512 --num_workers 1 --prior_model vit-lp --pvit --score cross_entropy

modify the --score argument. The available options are:

• cross_entropy for CE
• KL for KL divergence
• dis for ED

modify the --prior_model argument to evaluate with different prior models.

To simply reproduce the results presented in the paper:

1. Download the saved model outputs . If you do this, you do not need to download the dataset.
2. Place the unzipped folder in the root directory.