By Xintao Wang, Ke Yu, Chao Dong, Chen Change Loy.

This repo only provides simple testing codes - original torch version used in the paper and a pytorch version. For full training and testing codes, please refer to BasicSR.

@InProceedings{wang2018sftgan,
    author = {Wang, Xintao and Yu, Ke and Dong, Chao and Loy, Chen Change},
    title = {Recovering realistic texture in image super-resolution by deep spatial feature transform},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June},
    year = {2018}
}

1. Quick Test
2. Spatial Feature Modulation
3. Semantic Categorical Prior
4. OST dataset

It provides Torch and PyTorch versions. Recommend the PyTorch version.

• Python 3
• PyTorch >= 0.4.0
• Python packages: pip install numpy opencv-python

• Torch
• Other torch dependencies, e.g. nngraph, paths, image (install them by luarocks install xxx)

Note that the SFTGAN model is limited to some outdoor scenes. It is an unsatisfying limitation that we need to relax in future.

1. Clone this github repo.

git clone https://github.com/xinntao/SFTGAN
cd SFTGAN

2. There are two sample images in the ./data/samples folder.
3. Download pretrained models from Google Drive or Baidu Drive. Please see model list for more details.
4. First run segmentation test.

[PyTorch]

cd pytorch_test
python test_segmentation.py

[Torch]

cd torch_test
th test_segmentation.lua

The segmentation results are then in ./data with _segprob, _colorimg, _byteimg suffix.

5. Run sftgan test.

[PyTorch]

python test_sftgan.py.

[Torch]

th test_sftgan.lua

The results are in then in ./data with _result suffix.

SFT - Spatial Feature Transform (Modulation).

A Spatial Feature Transform (SFT) layer has been proposed to efficiently incorporate the categorical conditions into a CNN network.

There is a fantastic blog explaining the widely-used feature modulation operation distill - Feature-wise transformations.

We have explored the use of semantic segmentation maps as categorical prior for SR.

• Outdoor Scene Train/Test

OST300 300 test images of outdoor scences

Download the OST dataset from Google Drive or Baidu Drive.

May try HandyViewer - an image viewer that you can switch image with a fixed zoom ratio, easy for comparing image details.