This is the codebase for our paper TCNet: Continuous Sign Language Recognition from Trajectories and Correlated Regions, the repository is based on TLP and CorrNet, as we use the same feature extraction backbone, sequential modeling and classifier.
The required packages are in the file requirements.txt, and you can run the following command to install the environment
conda create --name TCNet python=3.8 -y
conda activate TCNet
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 -c pytorch
pip install -r requirements.txt
- The above commands are for reference only, please configure your own environment according to your needs.
- We recommend installing
PyTorch >= 1.12.0, which may greatly reduce the GPU memory usage. - It is recommended to install
timm == 0.4.12, because some of the APIs we use are deprecated in the latest version of timm. - We have supported training with
PyTorch 2.0, but it has not been fully tested.
We read and process the same way as CorrNet, but with a different convention for the format of the data list file.
Run the following command to process the image sequence
cd ./preprocess
python data_preprocess.py --process-image --multiprocessing
Run the following command to process the image sequence
cd ./preprocess
python data_preprocess-T.py --process-image --multiprocessing
Run the following command to process the image sequence
cd ./preprocess
python data_preprocess-CSL.py --process-image --multiprocessing
Run the following command to process the image sequence
cd ./preprocess
python data_preprocess-CSL-Daily.py --process-image --multiprocessing
Run the following command to train the model from scratch
python main.py --device your_device