PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

Related tags

Deep Learning HAN
Overview

HAN

PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

This repository is for HAN introduced in the following paper

Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu, "Single Image Super-Resolution via a Holistic Attention Network", ECCV 2020, arxiv

The code is built on RCAN (PyTorch) and tested on Ubuntu 16.04/18.04 environment (Python3.6, PyTorch_0.4.0, CUDA8.0, cuDNN5.1) with Titan X/1080Ti/Xp GPUs.

Contents


  1. Introduction
  2. Train
  3. Test
  4. Acknowledgements

Introduction

Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-of-the-art single image super- resolution approaches.

Train Prepare training data Download DIV2K training data (800 training + 100 validtion images) from DIV2K dataset.

Begin to train

(optional) Download models for our paper and place them in '/HAN/experiment/HAN'. All the models (BIX2/3/4/8, BDX3) can be downloaded from GoogleDrive. You can use scripts in file 'demo.sh' to train models for our paper.

BI, scale 2, 3, 4, 8
#HAN BI model (x2)

python main.py --template HAN --save HANx2 --scale 2 --reset --save_results --patch_size 96 --pre_train ../experiment/model/RCAN_BIX2.pt

#HAN BI model (x3)

python main.py --template HAN --save HANx3 --scale 3 --reset --save_results --patch_size 144 --pre_train ../experiment/model/RCAN_BIX2.pt

#HAN BI model (x4)

python main.py --template HAN --save HANx4 --scale 4 --reset --save_results --patch_size 192 --pre_train ../experiment/model/RCAN_BIX2.pt

#HAN BI model (x8)

python main.py --template HAN --save HANx8 --scale 8 --reset --save_results --patch_size 384 --pre_train ../experiment/model/RCAN_BIX2.pt

Begin to Test

Quick start

Download models for our paper and place them in '/experiment/HAN'.

Cd to '/HAN/src', run the following scripts.
#test
python main.py --template HAN --data_test Set5+Set14+B100+Urban100+Manga109 --data_range 801-900 --scale 2 --pre_train ../experiment/HAN/HAN_BIX2.pt --test_only --save HANx2_test --save_results

All the models (BIX2/3/4/8, BDX3) can be downloaded from GoogleDrive.

The whole test pipeline

1.Prepare test data.

Place the original test sets in '/dataset/x4/test'.

Run 'Prepare_TestData_HR_LR.m' in Matlab to generate HR/LR images with different degradation models.

2.Conduct image SR.

See Quick start

3.Evaluate the results.

Run 'Evaluate_PSNR_SSIM.m' to obtain PSNR/SSIM values for paper.

Acknowledgements

This code is built on RCAN. We thank the authors for sharing their codes of RCAN PyTorch version.

Comments
  • Pretrained models link (GoogleDrive) not accessible.

    Pretrained models link (GoogleDrive) not accessible.

    Hello! Thank you for taking the time to upload your work.

    I was hoping to test your model's results without starting from scratch but the google drive link for the pretrained weights seems to require approved access instead of being open.

    opened by kjerk 4
  • PSNR is not updating during training

    PSNR is not updating during training

    Screenshot 2021-10-25 124543

    During training for scale x4 PSNR value is not updating same is the case with scale 2 and others. Can someone guide me in this regard that what mistake I am doing?

    opened by aatiqa-ghazali 1
  • I want save SR image for my latest weight

    I want save SR image for my latest weight

    hello your model is good performance for my dataset!! if i want save SR image but i don't know how to save SR images if you learn to me for how to save SR images? i have best weight for your model ,and i want use this weight for save SR images . i use only python for pytorch not use matlab .thanks!!

    opened by cf0303 0
  • 'load_state_dict' method just pass some params of the pretrained

    'load_state_dict' method just pass some params of the pretrained

    Your pre-trained ckpt and model prams have different param keys for some layers.

    Pretrained state-dict has the following keys:

    ga.gamma
    ga.conv.weight
    ga.conv.bias
    da.gamma
    

    But the current model has different names for those layers:

    csa.gamma
    csa.conv.weight
    csa.conv.bias
    la.gamma
    

    So, the 'load_state_dict' function from the model class does not work in a proper way, just ignoring those layers.

    Is it OK for the reproducibility of your model?

    opened by toyaji 0
  • Super-Resolved images

    Super-Resolved images

    Hi,

    Can you provide a link that contains the Super-Resolved images (of different scales) from the benchmark datasets? Like a zip file or something similar?

    opened by Jee-King 1
Owner
五维空间
五维空间
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 26 Jul 30, 2022
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1.2k Sep 26, 2022
PyTorch implementation of ECCV 2020 paper "Foley Music: Learning to Generate Music from Videos "

Foley Music: Learning to Generate Music from Videos This repo holds the code for the framework presented on ECCV 2020. Foley Music: Learning to Genera

Chuang Gan 30 Sep 9, 2022
Code for the paper: Adversarial Training Against Location-Optimized Adversarial Patches. ECCV-W 2020.

Adversarial Training Against Location-Optimized Adversarial Patches arXiv | Paper | Code | Video | Slides Code for the paper: Sukrut Rao, David Stutz,

Sukrut Rao 29 Aug 30, 2022
Code for the paper "Improving Vision-and-Language Navigation with Image-Text Pairs from the Web" (ECCV 2020)

Improving Vision-and-Language Navigation with Image-Text Pairs from the Web Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh

Arjun Majumdar 42 Sep 15, 2022
Code for ECCV 2020 paper "Contacts and Human Dynamics from Monocular Video".

Contact and Human Dynamics from Monocular Video This is the official implementation for the ECCV 2020 spotlight paper by Davis Rempe, Leonidas J. Guib

Davis Rempe 188 Sep 23, 2022
SNE-RoadSeg in PyTorch, ECCV 2020

SNE-RoadSeg Introduction This is the official PyTorch implementation of SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentati

null 229 Sep 5, 2022
Unofficial PyTorch implementation of "RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving" (ECCV 2020)

RTM3D-PyTorch The PyTorch Implementation of the paper: RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (ECCV 2020

Nguyen Mau Dzung 265 Sep 5, 2022
Code for Towards Streaming Perception (ECCV 2020) :car:

sAP — Code for Towards Streaming Perception ECCV Best Paper Honorable Mention Award Feb 2021: Announcing the Streaming Perception Challenge (CVPR 2021

Martin Li 83 Sep 8, 2022
Source code for "Progressive Transformers for End-to-End Sign Language Production" (ECCV 2020)

Progressive Transformers for End-to-End Sign Language Production Source code for "Progressive Transformers for End-to-End Sign Language Production" (B

null 50 Sep 6, 2022
[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.

Stable Head Pose Estimation and Landmark Regression via 3D Dense Face Reconstruction Reimplementation of (ECCV 2020) Towards Fast, Accurate and Stable

Remilia Scarlet 201 Sep 15, 2022
1st Place Solution to ECCV-TAO-2020: Detect and Represent Any Object for Tracking

Instead, two models for appearance modeling are included, together with the open-source BAGS model and the full set of code for inference. With this code, you can achieve around [email protected] with TAO test set (based on our estimation).

null 80 Aug 22, 2022
Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020)

Causality In Traffic Accident (Under Construction) Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020) Overview Data Prepa

Tackgeun 19 Sep 23, 2022
git《Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction》(ECCV 2020) GitHub:

Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction Code for the ECCV 2020 paper by Yiming Qian and Yasutaka Furukawa Getting

null 35 Jul 8, 2022
dataset for ECCV 2020 "Motion Capture from Internet Videos"

Motion Capture from Internet Videos Motion Capture from Internet Videos Junting Dong*, Qing Shuai*, Yuanqing Zhang, Xian Liu, Xiaowei Zhou, Hujun Bao

ZJU3DV 94 Sep 9, 2022
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 43 Sep 16, 2022
[ECCV 2020] Gradient-Induced Co-Saliency Detection

Gradient-Induced Co-Saliency Detection Zhao Zhang*, Wenda Jin*, Jun Xu, Ming-Ming Cheng ⭐ Project Home » The official repo of the ECCV 2020 paper Grad

Zhao Zhang 33 Jun 14, 2022
Sign Language Translation with Transformers (COLING'2020, ECCV'20 SLRTP Workshop)

transformer-slt This repository gathers data and code supporting the experiments in the paper Better Sign Language Translation with STMC-Transformer.

Kayo Yin 97 Jul 31, 2022
IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)

This repo is the official implementation of our paper "Instance Adaptive Self-training for Unsupervised Domain Adaptation". The purpose of this repo is to better communicate with you and respond to your questions. This repo is almost the same with Another-Version, and you can also refer to that version.

CVSM Group -  email: czhu@bupt.edu.cn 85 Jul 30, 2022