Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Overview

论文名称:Image Super-Resolution Using Very Deep Residual Channel Attention Networks

目录

1. 简介
2. 数据集和复现精度
3. 开始使用
4. 代码结构与详细说明
5. 复现模型超分效果
5. 复现模型相关信息

1. 简介

本项目复现的论文是Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu, 发表在ECCV 2018上的论文。 作者提出了一个深度残差通道注意力网络(RCAN)。特别地,作者设计了一个残差中的残差(RIR)结构来构造深层网络,每个 RIR 结构由数个残差组(RG)以及长跳跃连接(LSC)组成,每个 RG 则包含一些残差块和短跳跃连接(SSC)。RIR 结构允许丰富的低频信息通过多个跳跃连接直接进行传播,使主网络专注于学习高频信息。此外,我们还提出了一种通道注意力机制(CA),通过考虑通道之间的相互依赖性来自适应地重新调整特征。

论文: 《Image Super-Resolution Using Very Deep Residual Channel Attention Networks》

参考repo: RCAN

在此非常感谢yulunzhang、MaFuyan、joaoherrera等人贡献的RCAN,提高了本项目的复现效率。

aistudio体验教程: 使用PaddleGAN复现RCAN

2. 数据集和复现精度

本项目所用到的训练集以及测试集包括相应的下载地址如下:

Name 数据集 数据描述 下载
2K Resolution DIV2K proposed in NTIRE17 (800 train and 100 validation) official website
Classical SR Testing Set5 Set5 test dataset Google Drive / Baidu Drive
Classical SR Testing Set14 Set14 test dataset Google Drive / Baidu Drive

数据集DIV2K, Set5 和 Set14 的组成形式如下:

  PaddleGAN
    ├── data
        ├── DIV2K
              ├── DIV2K_train_HR
              ├── DIV2K_train_LR_bicubic
              |    ├──X2
              |    ├──X3
              |    └──X4
              ├── DIV2K_valid_HR
              ├── DIV2K_valid_LR_bicubic
        ├── Set5
              ├── GTmod12
              ├── LRbicx2
              ├── LRbicx3
              ├── LRbicx4
              └── original
        ├── Set14
              ├── GTmod12
              ├── LRbicx2
              ├── LRbicx3
              ├── LRbicx4
              └── original
            ...

论文中模型(torch框架下训练)在Set14与Set5精度与使用paddle复现模型的精度对比:

框架 Set14
paddle 29.02 / 0.7910
torch 28.98 / 0.7910

Paddle模型(.pdparams)下载

模型 数据集 下载地址 提取码
rcan_x4 DIV2K rcan_x4 1ry9

3. 开始使用

3.1 准备环境

  • 硬件: Tesla V100 * 1
  • 框架:
    • PaddlePaddle >= 2.1.0
    • tqdm
    • PyYAML>=5.1
    • scikit-image>=0.14.0
    • scipy>=1.1.0
    • opencv-python
    • imageio==2.9.0
    • imageio-ffmpeg
    • librosa
    • numba==0.53.1
    • natsort
    • munch
    • easydict

将本项目git clone之后进入项目,使用pip install -r requirements.txt安装依赖即可。

3.2 快速开始

第一步:克隆本项目

# clone this repo
git clone https://github.com/kongdebug/RCAN-Paddle.git
cd RCAN-Paddle

第二步:安装依赖项

pip install -r requirements.txt

第三步:开始训练

单卡训练:

python -u tools/main.py --config-file configs/rcan_x4_div2k.yaml

由于本项目没有使用多卡训练,故不提供相关代码。 如使您想使用自己的数据集以及测试集,需要在配置文件中修改数据集为您自己的数据集。

如果训练断掉,想接着训练:

python -u tools/main.py --config-file configs/rcan_x4_div2k.yaml --resume ${PATH_OF_CHECKPOINT}

第四步:测试

  • 输出预测图像
    • 可以通过第二部分拿到paddle复现的模型,放入一个文件夹中,运行如下程序,得到模型的测试结果
    • Fig/visual文件夹中有预测结果,可直接用于精度评价
python -u tools/main.py --config-file configs/rcan_x4_div2k.yaml --evaluate-only --load ${PATH_OF_WEIGHT}
  • 对预测图像精度评价
    • 运行以上代码后,在output_dir文件夹中得到模型得预测结果,然后运行如下代码进行精度评定。注:--gt_dir与 output_dir两个参数得设置需要对应自己的实际路径。
python  tools/cal_psnr_ssim.py  --gt_dir data/Set14/GTmod12 --output_dir output_dir/rcan_x4_div2k*/visual_test

4. 代码结构与详细说明

4.1 代码结构

├─applications                          
├─benchmark                        
├─deploy                         
├─configs                          
├─data                        
├─output_dir                         
├─ppgan       
├─tools
├─test_tipc
├─Figs
│  README_cn.md                     
│  requirements.txt                      
│  setup.py                                         

4.2 结构说明

本项目基于PaddleGAN开发。configs文件夹中的rcan_x4_div2k.yaml是训练的配置文件,格式沿袭PaddleGAN中的SISR任务,参数设置与论文一致。data文件夹存放训练数据以及 测试数据。output_dir文件夹存放运行过程中输出的文件,一开始为空。test_tipc是用于导出模型预测,以及 TIPC测试的文件夹。

4.3 导出模型部署

  • 训练结束后得到rcan_checkpoint.pdparams文件,需要进行导出inference的步骤。
python3.7 tools/export_model.py -c configs/rcan_x4_div2k.yaml --inputs_size="-1,3,-1,-1" --load output_dir/rcan_checkpoint.pdparams --output_dir ./test_tipc/output/rcan_x4
  • 得到以上模型文件之后,基于PaddleInference对待预测推理的测试数据进行预测。
    • 将上一步导出的inference文件(.pdmodel、.pdiparams以及.pdiparams.info )均放入test_tipc/output/rcan_x4文件夹,注:文件名称均为basesrmodel_generator
    • 运行以下命令,在test_tipc/output/文件夹中得到预测结果
python3.7 tools/inference.py --model_type rcan --seed 123 -c configs/rcan_x4_div2k.yaml --output_path test_tipc/output/ --device=gpu --model_path=./test_tipc/output/rcan_x4/basesrmodel_generator

4.5 TIPC测试支持

test_tipc文件夹下文结构

test_tipc/
├── configs/  # 配置文件目录
    ├── rcan    
        ├── train_infer_python.txt      # 测试Linux上python训练预测(基础训练预测)的配置文件
        ├── train_infer_python_resume.txt      # 加载模型的(基础训练预测)的配置文件
├── output/   # 预测结果
├── common_func.sh    # 基础功能程序
├── prepare.sh                        # 需要的数据和模型下载
├── test_train_inference_python.sh    # 测试python训练预测的主程序
├── readme.md                # TIPC基础链接测试需要安装的依赖说明

注意: 本项目仅提供TIPC基础测试链条中模式lite_train_lite_infer的代码与文档。运行之前先使用vim查看.sh文件的filemode,需要为“filemode=unix"格式。

如果没有准备训练数据,可以运行prepare.sh下载训练数据DIV2K,然后对其解压,调整文件组织如第二部分所示; 如果已经准备好数据,运行如下命令完成TIPC基础测试:

  • 从头开始:
 bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/rcan/train_infer_python.txt 'lite_train_lite_infer'

这里需要注意,这里测试训练时所用的配置文件为configs文件夹下专门为从头开始的lite_train_lite_infer模式设置 的rcan_x4_div2k_tipc.yaml文件,没有加载训练好的模型而是从头训练,所以loss会很高。运行得到的结果在output 文件夹中,项目中该文件夹已放入先前运行得到的日志文件。

  • 加载已训练模型:
    • 将下载的rcan_checkpoint.pdparams模型文件,放入output_dir文件夹下,并改名为iter_238000_checkpoint.pdparams
    • 这里测试需要用的configs文件夹下的rcan_x4_div2k.yaml文件以及train_infer_python_resume.txt文件
    • 运行以下命令:
bash test_tipc/test_train_inference_python.sh ./test_tipc/configs/rcan/train_infer_python_resume.txt 'lite_train_lite_infer'

按照”加载已训练模型“的命令运行之后,最后会得到inference预测的结果图以及精度评价,可以看到psnr与ssim均达标。

5.复现模型超分效果

低分辨率 超分重建后 高分辨率

6.复现模型相关信息

相关信息:

信息 描述
作者 不想科研的Key.L
日期 2021年11月
框架版本 PaddlePaddle==2.2.0
应用场景 图像超分
硬件支持 GPU、CPU
在线体验 notebook
You might also like...
PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

HAN PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network" This repository is for HAN introduced in the

【Arxiv】Exploring Separable Attention for Multi-Contrast MR Image Super-Resolution

SANet Exploring Separable Attention for Multi-Contrast MR Image Super-Resolution Dependencies numpy==1.18.5 scikit_image==0.16.2 torchvision==0.8.1 to

Unofficial pytorch implementation of the paper "Dynamic High-Pass Filtering and Multi-Spectral Attention for Image Super-Resolution"

DFSA Unofficial pytorch implementation of the ICCV 2021 paper "Dynamic High-Pass Filtering and Multi-Spectral Attention for Image Super-Resolution" (p

A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022)
A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022)

A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022) https://arxiv.org/abs/2203.09388 Jianqi Ma, Zheto

PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

 Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution
Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution

Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution Figure: Example visualization of the method and baseline as a

Official implementation of Unfolded Deep Kernel Estimation for Blind Image Super-resolution.

Unfolded Deep Kernel Estimation for Blind Image Super-resolution Hongyi Zheng, Hongwei Yong, Lei Zhang, "Unfolded Deep Kernel Estimation for Blind Ima

Official PyTorch implementation of the paper
Official PyTorch implementation of the paper "Deep Constrained Least Squares for Blind Image Super-Resolution", CVPR 2022.

Deep Constrained Least Squares for Blind Image Super-Resolution [Paper] This is the official implementation of 'Deep Constrained Least Squares for Bli

Implementation for our ICCV 2021 paper: Dual-Camera Super-Resolution with Aligned Attention Modules
Implementation for our ICCV 2021 paper: Dual-Camera Super-Resolution with Aligned Attention Modules

DCSR: Dual Camera Super-Resolution Implementation for our ICCV 2021 oral paper: Dual-Camera Super-Resolution with Aligned Attention Modules paper | pr

Owner
kongdebug
kongdebug
PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

About PyTorch 1.2.0 Now the master branch supports PyTorch 1.2.0 by default. Due to the serious version problem (especially torch.utils.data.dataloade

Sanghyun Son 2.1k Jan 1, 2023
Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution"

NTIRE2017 Super-resolution Challenge: SNU_CVLab Introduction This is our project repository for CVPR 2017 Workshop (2nd NTIRE). We, Team SNU_CVLab, (B

Bee Lim 625 Dec 30, 2022
Pytorch implementation of Deep Recursive Residual Network for Super Resolution (DRRN)

DRRN-pytorch This is an unofficial implementation of "Deep Recursive Residual Network for Super Resolution (DRRN)", CVPR 2017 in Pytorch. [Paper] You

yun_yang 192 Dec 12, 2022
PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction... Attention in Attention Network for Image Super-Resolution (A2N) This repository is an PyTorch implementation of the paper "Atten

Haoyu Chen 71 Dec 30, 2022
Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! Very tiny! Stock Market Financial Technical Analysis Python library . Quant Trading automation or cryptocoin exchange

MyTT Technical Indicators implemented in Python only using Numpy-Pandas as Magic - Very Very Fast! to Stock Market Financial Technical Analysis Python

dev 34 Dec 27, 2022
A very tiny, very simple, and very secure file encryption tool.

Picocrypt is a very tiny (hence "Pico"), very simple, yet very secure file encryption tool. It uses the modern ChaCha20-Poly1305 cipher suite as well

Evan Su 1k Dec 30, 2022
Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.

scc4onnx Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel

Katsuya Hyodo 16 Dec 22, 2022
Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

Channel Pruning for Accelerating Very Deep Neural Networks (ICCV'17)

Yihui He 1k Jan 3, 2023
PyTorch code for our paper "Image Super-Resolution with Non-Local Sparse Attention" (CVPR2021).

Image Super-Resolution with Non-Local Sparse Attention This repository is for NLSN introduced in the following paper "Image Super-Resolution with Non-

null 143 Dec 28, 2022