Important Notes:

1. This are examples for fine-tuning the whole pretrained models, and not feature extraction.
2. If you are intersted on using feature extraction as mentioned on the paper, please refer to the embeding section .
3. You must not freeze the pretrained models, otherwise, you will get worse results.
4. Applying hyperparameter search (learning rate, seed, number of epochs, etc.) should give you better results than the mentioned results.