Trend-Aware Supervision: On Learning Invariance for Semi-Supervised Facial Action Unit Intensity Estimation

Official implementation of "Trend-Aware Supervision: On Learning Invariance for Semi-Supervised Facial Action Unit Intensity Estimation" (AAAI 2024).

💡 Motivation

🎥 Application

Please refer to repo for more details.

🛠️ Usage

Dependencies

git clone https://github.com/echoanran/Trend_Aware_Supervision.git $INSTALL_DIR

• python >= 3.6
• torch >= 1.10.0
• requirements.txt

$ pip install -r requirements.txt

• torchlight

$ cd $INSTALL_DIR/torchlight
$ python setup.py install

• R

$ conda install r-base 
$ conda install rpy2
$ R
> install.packages('psych')
> install.packages('lme4')
> quit()

Data Preparation

Step 1: Download datasets

First, request for the access of the two AU benchmark datasets: BP4D and DISFA.

Step 2: Preprocess raw data

Preprocess the downloaded datasets using Dlib (related functions are provided in $INSTALL_DIR/au_lib/face_ops.py):

• Detect face and facial landmarks
• Align the cropped faces according to the computed coordinates of eye centers
• Resize faces to (256, 256)

Step 3: Split dataset

For BP4D, use the official spliting file in BP4D. For DISFA, split the subject IDs into 3 folds randomly for subject-exclusive 3-fold cross-validation (an example is provided in $INSTALL_DIR/data/split_disfa.txt)

Step 4: Generate feeder input files

Our dataloader $INSTALL_DIR/feeder/feeder_segment.py requires two data files (an example is given in $INSTALL_DIR/data/bp4d_example):

• label_path: the path to file which contains labels ('.pkl' data), [N, 1, num_class]
• image_path: the path to file which contains image paths ('.pkl' data), [N, 1]
• state_path: the path to file which contains states ('.pkl' data), [N, 1, num_class]
• trend_path: the path to file which contains trends ('.pkl' data), [N, 1, num_class]

Training

$ cd $INSTALL_DIR
$ python run.py

🔗 Citation

If you find this code useful for your research, please use the following BibTeX entry.

@inproceedings{chen2024trend,
  title={Trend-Aware Supervision: On Learning Invariance for Semi-Supervised Facial Action Unit Intensity Estimation},
  author={Chen, Yingjie and Zhang, JiaRui and Wang, Tao and Liang, Yun},
  journal={arXiv preprint arXiv:2503.08078},
  booktitle={AAAI},
  year={2024}}