PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

Related tags

Deep Learning HAN
Overview

HAN

PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

This repository is for HAN introduced in the following paper

Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu, "Single Image Super-Resolution via a Holistic Attention Network", ECCV 2020, arxiv

The code is built on RCAN (PyTorch) and tested on Ubuntu 16.04/18.04 environment (Python3.6, PyTorch_0.4.0, CUDA8.0, cuDNN5.1) with Titan X/1080Ti/Xp GPUs.

Contents


  1. Introduction
  2. Train
  3. Test
  4. Acknowledgements

Introduction

Informative features play a crucial role in the single image super-resolution task. Channel attention has been demonstrated to be effective for preserving information-rich features in each layer. However, channel attention treats each convolution layer as a separate process that misses the correlation among different layers. To address this problem, we propose a new holistic attention network (HAN), which consists of a layer attention module (LAM) and a channel-spatial attention module (CSAM), to model the holistic interdependencies among layers, channels, and positions. Specifically, the proposed LAM adaptively emphasizes hierarchical features by considering correlations among layers. Meanwhile, CSAM learns the confidence at all the positions of each channel to selectively capture more informative features. Extensive experiments demonstrate that the proposed HAN performs favorably against the state-of-the-art single image super- resolution approaches.

Train Prepare training data Download DIV2K training data (800 training + 100 validtion images) from DIV2K dataset.

Begin to train

(optional) Download models for our paper and place them in '/HAN/experiment/HAN'. All the models (BIX2/3/4/8, BDX3) can be downloaded from GoogleDrive. You can use scripts in file 'demo.sh' to train models for our paper.

BI, scale 2, 3, 4, 8
#HAN BI model (x2)

python main.py --template HAN --save HANx2 --scale 2 --reset --save_results --patch_size 96 --pre_train ../experiment/model/RCAN_BIX2.pt

#HAN BI model (x3)

python main.py --template HAN --save HANx3 --scale 3 --reset --save_results --patch_size 144 --pre_train ../experiment/model/RCAN_BIX2.pt

#HAN BI model (x4)

python main.py --template HAN --save HANx4 --scale 4 --reset --save_results --patch_size 192 --pre_train ../experiment/model/RCAN_BIX2.pt

#HAN BI model (x8)

python main.py --template HAN --save HANx8 --scale 8 --reset --save_results --patch_size 384 --pre_train ../experiment/model/RCAN_BIX2.pt

Begin to Test

Quick start

Download models for our paper and place them in '/experiment/HAN'.

Cd to '/HAN/src', run the following scripts.
#test
python main.py --template HAN --data_test Set5+Set14+B100+Urban100+Manga109 --data_range 801-900 --scale 2 --pre_train ../experiment/HAN/HAN_BIX2.pt --test_only --save HANx2_test --save_results

All the models (BIX2/3/4/8, BDX3) can be downloaded from GoogleDrive.

The whole test pipeline

1.Prepare test data.

Place the original test sets in '/dataset/x4/test'.

Run 'Prepare_TestData_HR_LR.m' in Matlab to generate HR/LR images with different degradation models.

2.Conduct image SR.

See Quick start

3.Evaluate the results.

Run 'Evaluate_PSNR_SSIM.m' to obtain PSNR/SSIM values for paper.

Acknowledgements

This code is built on RCAN. We thank the authors for sharing their codes of RCAN PyTorch version.

Issues
  • Pretrained models link (GoogleDrive) not accessible.

    Pretrained models link (GoogleDrive) not accessible.

    Hello! Thank you for taking the time to upload your work.

    I was hoping to test your model's results without starting from scratch but the google drive link for the pretrained weights seems to require approved access instead of being open.

    opened by kjerk 4
  • What do you mean by 'Set20'?

    What do you mean by 'Set20'?

    Set20 = set5+set14???

    opened by Senwang98 2
  • PSNR is not updating during training

    PSNR is not updating during training

    Screenshot 2021-10-25 124543

    During training for scale x4 PSNR value is not updating same is the case with scale 2 and others. Can someone guide me in this regard that what mistake I am doing?

    opened by aatiqa-ghazali 1
  • question about your paper

    question about your paper

    Hi, Thanks for your nice work firstly! After reading your paper, I found LAM looks like DANet. Have you been inspired by DANet?

    opened by Senwang98 0
  • How to merge HAN into EDSR project?

    How to merge HAN into EDSR project?

    It seems that HAN can be merged into EDSR-pytorch. Just copy han.py to EDSR?

    opened by Senwang98 0
  • 'load_state_dict' method just pass some params of the pretrained

    'load_state_dict' method just pass some params of the pretrained

    Your pre-trained ckpt and model prams have different param keys for some layers.

    Pretrained state-dict has the following keys:

    ga.gamma
    ga.conv.weight
    ga.conv.bias
    da.gamma
    

    But the current model has different names for those layers:

    csa.gamma
    csa.conv.weight
    csa.conv.bias
    la.gamma
    

    So, the 'load_state_dict' function from the model class does not work in a proper way, just ignoring those layers.

    Is it OK for the reproducibility of your model?

    opened by toyaji 0
  • How to prepare test data and get performance indices?

    How to prepare test data and get performance indices?

    Where are Prepare_TestData_HR_LR.m and Evaluate_PSNR_SSIM.m?

    opened by yangbincheng 1
  • Super-Resolved images

    Super-Resolved images

    Hi,

    Can you provide a link that contains the Super-Resolved images (of different scales) from the benchmark datasets? Like a zip file or something similar?

    opened by Jee-King 1
  • An question in your code

    An question in your code

    energy_new = torch.max(energy, -1, keepdim=True)[0].expand_as(energy)-energy what's meaning for this line?

    opened by fantasysponge 0
  • HAN training is unstable

    HAN training is unstable

    when i training your han model, i found the loss exploded and model collapsed! Have your met this?or can you give me some guidence? image

    opened by smiler96 9
Owner
五维空间
五维空间
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 22 Jan 15, 2022
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1k Jan 12, 2022
PyTorch implementation of ECCV 2020 paper "Foley Music: Learning to Generate Music from Videos "

Foley Music: Learning to Generate Music from Videos This repo holds the code for the framework presented on ECCV 2020. Foley Music: Learning to Genera

Chuang Gan 20 Nov 5, 2021
Code for the paper: Adversarial Training Against Location-Optimized Adversarial Patches. ECCV-W 2020.

Adversarial Training Against Location-Optimized Adversarial Patches arXiv | Paper | Code | Video | Slides Code for the paper: Sukrut Rao, David Stutz,

Sukrut Rao 19 Dec 11, 2021
Code for the paper "Improving Vision-and-Language Navigation with Image-Text Pairs from the Web" (ECCV 2020)

Improving Vision-and-Language Navigation with Image-Text Pairs from the Web Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh

Arjun Majumdar 28 Dec 29, 2021
Code for ECCV 2020 paper "Contacts and Human Dynamics from Monocular Video".

Contact and Human Dynamics from Monocular Video This is the official implementation for the ECCV 2020 spotlight paper by Davis Rempe, Leonidas J. Guib

Davis Rempe 135 Jan 12, 2022
SNE-RoadSeg in PyTorch, ECCV 2020

SNE-RoadSeg Introduction This is the official PyTorch implementation of SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentati

null 189 Jan 7, 2022
Unofficial PyTorch implementation of "RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving" (ECCV 2020)

RTM3D-PyTorch The PyTorch Implementation of the paper: RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (ECCV 2020

Nguyen Mau Dzung 249 Dec 17, 2021
Code for Towards Streaming Perception (ECCV 2020) :car:

sAP — Code for Towards Streaming Perception ECCV Best Paper Honorable Mention Award Feb 2021: Announcing the Streaming Perception Challenge (CVPR 2021

Martin Li 70 Jan 1, 2022
Source code for "Progressive Transformers for End-to-End Sign Language Production" (ECCV 2020)

Progressive Transformers for End-to-End Sign Language Production Source code for "Progressive Transformers for End-to-End Sign Language Production" (B

null 39 Jan 16, 2022
[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.

Stable Head Pose Estimation and Landmark Regression via 3D Dense Face Reconstruction Reimplementation of (ECCV 2020) Towards Fast, Accurate and Stable

Remilia Scarlet 163 Jan 18, 2022
1st Place Solution to ECCV-TAO-2020: Detect and Represent Any Object for Tracking

Instead, two models for appearance modeling are included, together with the open-source BAGS model and the full set of code for inference. With this code, you can achieve around [email protected] with TAO test set (based on our estimation).

null 74 Nov 15, 2021
Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020)

Causality In Traffic Accident (Under Construction) Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020) Overview Data Prepa

Tackgeun 15 Oct 27, 2021
git《Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction》(ECCV 2020) GitHub:

Learning Pairwise Inter-Plane Relations for Piecewise Planar Reconstruction Code for the ECCV 2020 paper by Yiming Qian and Yasutaka Furukawa Getting

null 30 Dec 18, 2021
dataset for ECCV 2020 "Motion Capture from Internet Videos"

Motion Capture from Internet Videos Motion Capture from Internet Videos Junting Dong*, Qing Shuai*, Yuanqing Zhang, Xian Liu, Xiaowei Zhou, Hujun Bao

ZJU3DV 82 Dec 28, 2021
《Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglement》(ECCV 2020) GitHub: [fig9]

Unsupervised 3D Human Pose Representation [Paper] The implementation of our paper Unsupervised 3D Human Pose Representation with Viewpoint and Pose Di

null 39 Dec 15, 2021
[ECCV 2020] Gradient-Induced Co-Saliency Detection

Gradient-Induced Co-Saliency Detection Zhao Zhang*, Wenda Jin*, Jun Xu, Ming-Ming Cheng ⭐ Project Home » The official repo of the ECCV 2020 paper Grad

Zhao Zhang 32 Dec 15, 2021
Sign Language Translation with Transformers (COLING'2020, ECCV'20 SLRTP Workshop)

transformer-slt This repository gathers data and code supporting the experiments in the paper Better Sign Language Translation with STMC-Transformer.

Kayo Yin 81 Jan 17, 2022
IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)

This repo is the official implementation of our paper "Instance Adaptive Self-training for Unsupervised Domain Adaptation". The purpose of this repo is to better communicate with you and respond to your questions. This repo is almost the same with Another-Version, and you can also refer to that version.

CVSM Group -  email: czhu@bupt.edu.cn 84 Dec 13, 2021