Improving Knowledge Distillation

Improving KD via reducing regularization.

To train

install requirments.txt
run python train.py with the following arguments :

• teacher
  
• student
  
• learning rate
  
• pruning rate
  
• pruning mode
  
• batch size
  
• epochs
  
• optimizer
  
• temperature
  
• etc ..