Multi-task

1. Learning without Forgetting [ECCV 2016] Zhizhong Li, Derek Hoiem.
   • Train multi-task model in no old dataset.
   • Distillation and fine-tuning.
     • task-specific layers
     • network expansion
     • lower lr
     • loss
2. Dynamic Task Prioritization for Multitask Learning [ECCV 2018] Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, Li Fei-Fei.
   • Using Focal Loss Control the prioritization of different tasks.
   • using KPIs(like acc) to evaluate the sample hard degree.
   • give hard sample higher priority.
3. SNR: Sub-Network Routing for Flexible Parameter Sharing in Multi-Task Learning [AAAI 2019] J Ma, Z Zhao, J Chen, A Li, L Hong, EH Chi.
   • flexible connection of previous block in different task.
   • like a binary encoding.
   • To learn the (0, 1) weight, they follow the Louizos's work which relaxation to smooth such objective functions so that can directly calculate the gradients of the distribution parameters.