This is the code for our paper on ACM MM 2024 " UniQ: Unified Decoder with Task-specific Queries for Efficient Scene Graph Generation"
-
Create a conda environment
conda create -n UniQ python==3.8.5
-
Activate the environment
conda activate UniQ
-
Install packages
h5py==3.10.0 imantics==0.1.12 easydict==1.11 scikit-learn==1.3.2 scipy==1.10.1 pandas==2.0.3 detectron2==0.6 torch==2.2.0+cu118 torchvision==0.17.0+cu118
We adopt a subset of Visual Genome VG150 that contains the most frequent 150 object classes and 50 predicate classes.
- Download Visual Genome
dataset/
└── vg/
├── VG_100K/
├── image_data.json
├── VG-SGG-dicts-with-attri.json
└── VG-SGG-with-attri.h5
-
Train unbiased UniQ (with
$\alpha = 0.07$ ,$\beta = 0.75$ ). Train unbiased UniQ with more number of query groups by settingMODEL.DETR.GROUP_DETR
to a costumed number.bash train_UniQ.sh
-
STS, STT, TST baselines can be trained with their corresponding config files in
configs/
.
-
Evaluate unbiased UniQ
bash test_UniQ.sh
-
Evaluate unbiased UniQ with top-k links by setting
MODEL.DETR.MATCHER_TOPK 3
@inproceedings{10.1145/3664647.3681542,
title = {UniQ: Unified Decoder with Task-specific Queries for Efficient Scene Graph Generation},
author = {Liao Xinyao and Wei Wei and Chen Dangyang and Fu Yuanyuan},
booktitle = {Proceedings of the 32nd ACM International Conference on Multimedia},
pages = {8815–8824},
year = {2024}
}
- This repository is built upon the Iterative Scene Graph Generation developed by Siddhesh Khandelwal and Leonid Sigal. Thanks for their extraordinary contribution to this field.
- This repository also extended DETR and Group DETR to SGG.