This repository provides a PyTorch implementation of the paper Temporal Relational Modeling with Self-Supervision for Action Segmentation.

Tested with:

• PyTorch 1.6.0
• Python 3.6.12

• Download the data folder, which contains the features and the ground truth labels. (~30GB). We use the same data with MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation
• Extract it so that you have the data folder in the same directory as main.py.
• To train the model run python main.py --action=train --dataset=DS --split=SP --num_stages=4 --num_layers=10 --num_f_maps=64 --df_size=3 where DS is breakfast, 50salads or gtea, and SP is the split number (1-5) for 50salads and (1-4) for the other datasets. num_stages, num_layers and df_size (order of graphs) are the hyper-parameters of the proposed model, which can be changed to other values.

Run python main.py --action=predict --dataset=DS --split=SP --num_stages=4 --num_layers=10 --num_f_maps=64 --df_size=3.

Run python eval.py --dataset=DS --split=SP --num_stages=4 --num_layers=10 --num_f_maps=64 --df_size=3.

If you use the code, please cite

D. Wang, H. Di, X. Li, and D. Dou.
Temporal Relational Modeling with Self-Supervision for Action Segmentation.
In AAAI Conference on Artificial Intelligence(AAAI), 2021

This code is borrowed or adapted from MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation. Thanks a lot for their great work!