MetaF2N: Blind Image Super-Resolution by Learning Efficient Model Adaptation from Faces

Getting Started- IPR_Project

Environment Setup

conda create -n metaf2n python=3.9
conda activate metaf2n
pip install -r requirements.txt

Trained Models

I provided the trained checkpoints in Google Drive. One can download them and save to the directory ./weights/Model1.

Updated trained model can be found in Google Drive. One can download them and save to the directory ./weights/Model2.

Moreover, the other models (pretrained GPEN, Real-ESRGAN and RetinaFace) we need for training and testing are also provided in Google Drive. One can download them and save to the directory ./weights.

Preparing Dataset

• Since I use tensoflow as our framework, I prepared our training data in the format of .tfrecord as Google Drive . One can download them and save to the directory ./datasets.

• Updated Preprocessed triaining data in the format of .tfrecord can be found in Google Drive . One can download them and save to the directory ./datasets.

• If you want to prepare the training data yourself, you can use the generate_tfrecord.py and change the parameters.

  python scripts/generate_tfrecord.py

• All the test data are provided as Google Drive. Each synthsized dataset has two subfolders (GT, LQ) you can also generate Face_LQ and Face_HQ using this:

  python generate_test_faces.py --input_dir input_dir --output_dir output_dir

• The final test data structure is like this:

  ├── datasets
  │   ├── CelebA_iid
  │   │   ├── Face_HQ
  │   │   ├── Face_LQ
  │   │   ├── GT
  │   │   └── LQ
  │   ├── CelebA_ood
  │   │   ├── Face_HQ
  │   │   ├── Face_LQ
  │   │   ├── GT
  │   │   └── LQ
  │   ├── FFHQ_iid
  │   │   ├── Face_HQ
  │   │   ├── Face_LQ
  │   │   ├── GT
  │   │   └── LQ
  │   ├── FFHQ_Multi_iid
  │   │   ├── Face_HQ
  │   │   ├── Face_LQ
  │   │   ├── GT
  │   │   └── LQ
  │   ├── FFHQ_Multi_ood
  │   │   ├── Face_HQ
  │   │   ├── Face_LQ
  │   │   ├── GT
  │   │   └── LQ
  │   ├── FFHQ_ood
  │   │   ├── Face_HQ
  │   │   ├── Face_LQ
  │   │   ├── GT
  │   │   └── LQ

Testing

To test the method, one can run,

CUDA_VISIBLE_DEVICES=0 test.py --input_dir input_dir --output_dir output_dir --face_dir face_dir --patch_size patch_size --patch_num_per_img patch_num_per_img --fine_tune_num fine_tune_num

Four parameters can be changed for flexible usage:

--input_dir # test LQ image path
--output_dir # save the results path
--face_dir # the path that contains Face_LQ (Cropped LQ Face Area) and Face_HQ (Retored HQ Face Area)
--patch_size # the patch size for the cropping of HQ face area
--patch_num_per_img # the number of patches for the cropping of HQ face area
--fine_tune_num # the steps of the inner loop fine-tuning

Training

To train MetaF2N, you can adjust the parameters in config.py and run, To train my updated model with the one can delete the existing train python file and change the name of train1.py file to train.py. and then after adjusting the parameters in config.py one can run the command given below,

python main.py --trial trial --step step --gpu gpu_id

Calculate Metrics

To calculate metrics of the results, one can run,

python calculate_metrics.py --result_dir result_dir --gt_dir gt_dir --fid_ref_dir fid_ref_dir