PyTorch code for our paper "Gated Multiple Feedback Network for Image Super-Resolution" (BMVC2019)

Related tags

Deep Learning GMFN
Overview

Gated Multiple Feedback Network for Image Super-Resolution

This repository contains the PyTorch implementation for the proposed GMFN [arXiv].

The framework of our proposed GMFN. The colored arrows among different time steps denote the multiple feedback connections. The high-level information carried by them helps low-level features become more representative.

Demo

Clone SRFBN as the backbone and satisfy its requirements.

Test

  1. Copy ./networks/gmfn_arch.py into SRFBN_CVPR19/networks/

  2. Download the pre-trained models from Google driver or Baidu Netdisk, unzip and place them into SRFBN_CVPR19/models.

  3. Copy ./options/test/ to SRFBN_CVPR19/options/test/.

  4. Run commands cd SRFBN_CVPR19 and one of followings for evaluation on Set5:

python test.py -opt options/test/test_GMFN_x2.json
python test.py -opt options/test/test_GMFN_x3.json
python test.py -opt options/test/test_GMFN_x4.json
  1. Finally, PSNR/SSIM values for Set5 are shown on your screen, you can find the reconstruction images in ./results.

To test GMFN on other standard SR benchmarks or your own images, please refer to the instruction in SRFBN.

Train

  1. Prepare the training set according to this (1-3).
  2. Modify ./options/train/train_GMFN.json by following the instructions in ./options/train/README.md.
  3. Run commands:
cd SRFBN_CVPR19
python train.py -opt options/train/train_GNFN.json
  1. You can monitor the training process in ./experiments.

  2. Finally, you can follow the test pipeline to evaluate the model trained by yourself.

Performance

Quantitative Results

Quantitative evaluation under scale factors x2, x3 and x4. The best performance is shown in bold and the second best performance is underlined.

More Qualitative Results (x4)

Acknowledgment

If you find our work useful in your research or publications, please consider citing:

@inproceedings{li2019gmfn,
    author = {Li, Qilei and Li, Zhen and Lu, Lu and Jeon, Gwanggil and Liu, Kai and Yang, Xiaomin},
    title = {Gated Multiple Feedback Network for Image Super-Resolution},
    booktitle = {The British Machine Vision Conference (BMVC)},
    year = {2019}
}

@inproceedings{li2019srfbn,
    author = {Li, Zhen and Yang, Jinglei and Liu, Zheng and Yang, Xiaomin and Jeon, Gwanggil and Wu, Wei},
    title = {Feedback Network for Image Super-Resolution},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year= {2019}
}
You might also like...
Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Lottery Jackpots Exist in Pre-trained Models (Paper Link) Requirements Python = 3.7.4 Pytorch = 1.6.1 Torchvision = 0.4.1 Reproduce the Experiment

The repository offers the official implementation of our paper in PyTorch.

Cloth Interactive Transformer (CIT) Cloth Interactive Transformer for Virtual Try-On Bin Ren1, Hao Tang1, Fanyang Meng2, Runwei Ding3, Ling Shao4, Phi

PyTorch implementations for our SIGGRAPH 2021 paper: Editable Free-viewpoint Video Using a Layered Neural Representation.
PyTorch implementations for our SIGGRAPH 2021 paper: Editable Free-viewpoint Video Using a Layered Neural Representation.

st-nerf We provide PyTorch implementations for our paper: Editable Free-viewpoint Video Using a Layered Neural Representation SIGGRAPH 2021 Jiakai Zha

PyTorch implementation of our ICCV2021 paper: StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation
PyTorch implementation of our ICCV2021 paper: StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation

StructDepth PyTorch implementation of our ICCV2021 paper: StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimat

Pytorch implementation for  our ICCV 2021 paper
Pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering".

TRAnsformer Routing Networks (TRAR) This is an official implementation for ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visu

This is the official pytorch implementation for our ICCV 2021 paper
This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

🌈 ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.
PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

PyTorch implementation for our NeurIPS 2021 Spotlight paper
PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short-Term Transformer for Online Action Detection".

Long Short-Term Transformer for Online Action Detection Introduction This is a PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short

Official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels".

WarPI The official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels". Run python main.py --corruption_type

Comments
  • Approximately how many epoches will reach the results in the paper (4x SR result)

    Approximately how many epoches will reach the results in the paper (4x SR result)

    Hi, liqilei After I have run about 700 epoches, the reult on val set is 32.41(highest result). I want to know if my training process seems to be problematic? How long did you reach 32.47 of SRFBN when you were training? How long does it take to reach 32.70? Thank you.

    opened by Senwang98 7
  • train error size not match

    train error size not match

    CUDA_VISIBLE_DEVICES=0 python train.py -opt options/train/train_GMFN.json I use celeba dataset train

    ===> Training Epoch: [1/1000]... Learning Rate: 0.000200 Epoch: [1/1000]: 0%| | 0/251718 [00:00<?, ?it/s] Traceback (most recent call last): File "train.py", line 131, in main() File "train.py", line 69, in main iter_loss = solver.train_step() File "/exp_sr/SRFBN/solvers/SRSolver.py", line 104, in train_step loss_steps = [self.criterion_pix(sr, split_HR) for sr in outputs] File "/exp_sr/SRFBN/solvers/SRSolver.py", line 104, in loss_steps = [self.criterion_pix(sr, split_HR) for sr in outputs] File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in call result = self.forward(*input, **kwargs) File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/modules/loss.py", line 87, in forward return F.l1_loss(input, target, reduction=self.reduction) File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/functional.py", line 1702, in l1_loss input, target, reduction) File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/functional.py", line 1674, in _pointwise_loss return lambd_optimized(input, target, reduction) RuntimeError: input and target shapes do not match: input [16 x 3 x 192 x 192], target [16 x 3 x 48 x 48] at /pytorch/aten/src/THCUNN/generic/AbsCriterion.cu:12

    opened by yja1 3
  • Not an Issue

    Not an Issue

    Hey @Paper99,

    Thanks for sharing your code! I wonder if it is possible to help with visualizing featuer-maps as you did in your paper figure 4.

    Thanks

    opened by Auth0rM0rgan 1
  • My training result with scale = 2

    My training result with scale = 2

    Hi, After I have trained the DIV2k, I get the final result(use best_ckp.pth to test):

    set5:38.16/0.9610
    set14:33.91/0.9203
    urban100:32.81/0.9349
    B100:32.30/0.9011
    manga109:39.01/0.9776
    

    It seems much lower than that in your paper.

    opened by Senwang98 6
Owner
Qilei Li
Qilei Li
PyTorch code for our paper "Image Super-Resolution with Non-Local Sparse Attention" (CVPR2021).

Image Super-Resolution with Non-Local Sparse Attention This repository is for NLSN introduced in the following paper "Image Super-Resolution with Non-

null 143 Dec 28, 2022
PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

HAN PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network" This repository is for HAN introduced in the

五维空间 140 Nov 23, 2022
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1.2k Dec 26, 2022
PyTorch code accompanying our paper on Maximum Entropy Generators for Energy-Based Models

Maximum Entropy Generators for Energy-Based Models All experiments have tensorboard visualizations for samples / density / train curves etc. To run th

Rithesh Kumar 135 Oct 27, 2022
Convolutional neural network web app trained to track our infant’s sleep schedule using our Google Nest camera.

Machine Learning Sleep Schedule Tracker What is it? Convolutional neural network web app trained to track our infant’s sleep schedule using our Google

g-parki 7 Jul 15, 2022
Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

CoProtector Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

Zhensu Sun 1 Oct 26, 2021
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 35 Dec 6, 2022
Official implementation of our CVPR2021 paper "OTA: Optimal Transport Assignment for Object Detection" in Pytorch.

OTA: Optimal Transport Assignment for Object Detection This project provides an implementation for our CVPR2021 paper "OTA: Optimal Transport Assignme

null 217 Jan 3, 2023
PyTorch implementation of the Deep SLDA method from our CVPRW-2020 paper "Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis"

Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis This is a PyTorch implementation of the Deep Streaming Linear Discriminant

Tyler Hayes 41 Dec 25, 2022
The official pytorch implementation of our paper "Is Space-Time Attention All You Need for Video Understanding?"

TimeSformer This is an official pytorch implementation of Is Space-Time Attention All You Need for Video Understanding?. In this repository, we provid

Facebook Research 1k Dec 31, 2022