Practical Single-Image Super-Resolution Using Look-Up Table

Related tags

Deep Learning SR-LUT
Overview

Practical Single-Image Super-Resolution Using Look-Up Table

[Paper]

Dependency

  • Python 3.6
  • PyTorch
  • glob
  • numpy
  • pillow
  • tqdm
  • tensorboardx

1. Training deep SR network

  1. Move into a directory.
cd ./1_Train_deep_model
  1. Prepare DIV2K training images into ./train.
  • HR images should be placed as ./train/DIV2K_train_HR/*.png.
  • LR images should be placed as ./train/DIV2K_train_LR_bicubic/X4/*.png.

  1. Set5 HR/LR validation png images are already included in ./val, or you can use other images.

  2. You may modify user parameters in L22 in ./Train_Model_S.py.

  3. Run.

python Train_Model_S.py
  1. Checkpoints will be saved in ./checkpoint/S.
  • Training log will be generated in ./log/S.

2. Transferring to LUT

  1. Move into a directory.
cd ./2_Transfer_to_LUT
  1. Modify user parameters in L9 in ./Transfer_Model_S.py.
  • Specify a saved checkpoint in the step 1, or you can use attached ./Model_S.pth.
  1. Run.
python Transfer_Model_S.py
  1. The resulting LUT will be saved like ./Model_S_x4_4bit_int8.npy.

3. Testing using LUT

  1. Move into a directory.
cd ./3_Test_using_LUT
  1. Modify user parameters in L17 in ./Test_Model_S.py.
  • Specify the generated LUT in the step 2, or use attached LUTs (npy files).

  1. Set5 HR/LR test images are already included in ./test, or you can use other images.

  2. Run.

python Test_Model_S.py      # Ours-S
python Test_Model_F.py      # Ours-F
python Test_Model_V.py      # Ours-V

  1. Resulting images will be saved in ./output_S_x4_4bit/*.png.

  2. We can reproduce the results of Table 6 in the paper, by modifying the variable SAMPLING_INTERVAL in L19 in Test_Model_S.py to range 3-8.

4. Testing on a smartphone

  1. Download SR-LUT.apk and install it.

  2. You can test Set14 images or other images.

SR-LUT Android app demo

BibTeX

@InProceedings{jo2021practical,
   author = {Jo, Younghyun and Kim, Seon Joo},
   title = {Practical Single-Image Super-Resolution Using Look-Up Table},
   booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
   month = {June},
   year = {2021}
}
Comments
  • Instead of the function rot90()

    Instead of the function rot90()

    I would like to ask if we can change the fill position instead of rotation by rotating the image 3 times on the right and rotating it back to the original position after interpolation. For example, the previous filling on the right side was changed to filling on the upper, lower, and left sides respectively, and then the interception was corresponding to the previous one. I tried not to rotate, but my experiment found that the result seemed to be wrong. Is there any special significance of rotation. millions of thanks.

    opened by djdll 6
  • Code of mobile app

    Code of mobile app

    Hi,

    Thank you for sharing your code with the community! I wanted to ask you if it's possible for you to share the implementation of the mobile application? Also which model is used in this later?

    Many thanks in advance! With best,

    opened by BounharAbdelaziz 4
  • hi，@yhjo9 According to the script provided by the warehouse, psnr has been oscillating around 27.5 (about 400,000 times) during the sr-lut training process. Is this the final result that the model can achieve? thanks

    hi，@yhjo9 According to the script provided by the warehouse, psnr has been oscillating around 27.5 (about 400,000 times) during the sr-lut training process. Is this the final result that the model can achieve? thanks

    hi，@yhjo9 According to the script provided by the warehouse, psnr has been oscillating around 27.5 (about 400,000 times) during the sr-lut training process. Is this the final result that the model can achieve? thanks

    opened by lei-xiong 1
  • The method to increase the receptive field

    The method to increase the receptive field

    Hi, The method to increase the receptive filed in this paper, actually, is rotating and padding. I'm trying to use the padding alone to increase the receptive filed, specifcally, padding the original image at (top or down) and (left or right) four times, and ensemble the results. (for example in the picture below, yellow area results will be ensembled). But the performance even obviously worse than the method that do not ensemble. It's strange, have you tried this method?

    image

    opened by gzhcv 1
  • Cannot reproduce the psnr value on Set14

    Cannot reproduce the psnr value on Set14

    Hi, i tested the psnr value on Set14 using the lut provided in this repo Model_S_x4_4bit_int8.npy, and get the value 26.41, which is different from paper that is 27.01. I used python package PIL to generate low resoultion image of Set14 (bicubic mode).

    So i'm curious about why there is such a big difference.

    Thanks for your sharing, hope for your reply

    opened by gzhcv 1
  • How to parallel execution in VS2017？

    How to parallel execution in VS2017？

    Thank you for your great work at first.

    Now we are planing to run SR-LUT on a PC(i7 CPU), we are considering to convert Python project (inference part) to VS2017.

    I think there will be no big problem for python converting to C++, but we concern the speed issue.

    I studied issue #1，and know that we should use parallel execution to improve the speed. for Android, it is Stream API. But for VS2017 on PC. would you please give me some suggestion on how to do parallel execution ? Is multipe thread a good solution?

    Thank you in advance.

    opened by Zhaoxuyu 1
  • Can you provide the source code of SR-LUT.apk? Thanks.

    Can you provide the source code of SR-LUT.apk? Thanks.

    Hi @yhjo09 I think your SR-LUT method is very good.Can you provide the source code of SR-LUT.apk?I am very eager to learn about it.Thank you very much.

    opened by winnerlqy 3
  • Could this LUT model run on GPU?

    Could this LUT model run on GPU?

    Hi @yhjo09 I wanna to know that this LUT model(.npy) is run on CPU or GPU?
    if it is run on CPU, it is possiable to run on GPU for fast inference?

       Thank you very much! ^_^
    opened by xiaoxiongli 1
Owner
Younghyun Jo
Younghyun Jo
GitHub
pytorch implementation for Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arXiv:1609.04802

PyTorch SRResNet Implementation of Paper: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"(https://arxiv.org/abs

Jiu XU 436 Jan 9, 2023
PyTorch code for 'Efficient Single Image Super-Resolution Using Dual Path Connections with Multiple Scale Learning'

Efficient Single Image Super-Resolution Using Dual Path Connections with Multiple Scale Learning This repository is for EMSRDPN introduced in the foll

null 7 Feb 10, 2022
PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

HAN PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network" This repository is for HAN introduced in the

五维空间 140 Nov 23, 2022
PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

About PyTorch 1.2.0 Now the master branch supports PyTorch 1.2.0 by default. Due to the serious version problem (especially torch.utils.data.dataloade

Sanghyun Son 2.1k Jan 1, 2023
Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution"

NTIRE2017 Super-resolution Challenge: SNU_CVLab Introduction This is our project repository for CVPR 2017 Workshop (2nd NTIRE). We, Team SNU_CVLab, (B

Bee Lim 625 Dec 30, 2022
Augmentation for Single-Image-Super-Resolution

SRAugmentation Augmentation for Single-Image-Super-Resolution Implimentation CutBlur Cutout CutMix Cutup CutMixup Blend RGBPermutation Identity OneOf

Yubo 6 Jun 27, 2022
Single Image Super-Resolution (SISR) with SRResNet, EDSR and SRGAN

Single Image Super-Resolution (SISR) with SRResNet, EDSR and SRGAN Introduction Image super-resolution (SR) is the process of recovering high-resoluti

null 8 Apr 15, 2022
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021) This repository is the official P

Jingyun Liang 159 Dec 30, 2022
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021) This repository is the official P

Jingyun Liang 159 Dec 30, 2022
Implementation of paper: "Image Super-Resolution Using Dense Skip Connections" in PyTorch

SRDenseNet-pytorch Implementation of paper: "Image Super-Resolution Using Dense Skip Connections" in PyTorch (http://openaccess.thecvf.com/content_ICC

wxy 114 Nov 26, 2022
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1.2k Dec 26, 2022
An unofficial implementation of "Unpaired Image Super-Resolution using Pseudo-Supervision." CVPR2020

UnpairedSR An unofficial implementation of "Unpaired Image Super-Resolution using Pseudo-Supervision." CVPR2020 turn RCAN(modified) --> xmodel(xilinx

JiaKui Hu 10 Oct 28, 2022
Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

kongdebug 14 Oct 14, 2022
Using image super resolution models with vapoursynth and speeding them up with TensorRT

vs-RealEsrganAnime-tensorrt-docker Using image super resolution models with vapoursynth and speeding them up with TensorRT. Also a docker image since

null 4 Aug 23, 2022
A framework for joint super-resolution and image synthesis, without requiring real training data

SynthSR This repository contains code to train a Convolutional Neural Network (CNN) for Super-resolution (SR), or joint SR and data synthesis. The met

null 83 Jan 1, 2023
Repository for "Exploring Sparsity in Image Super-Resolution for Efficient Inference", CVPR 2021

SMSR Reposity for "Exploring Sparsity in Image Super-Resolution for Efficient Inference" [arXiv] Highlights Locate and skip redundant computation in S

Longguang Wang 225 Dec 26, 2022
MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution (CVPR2021)

MASA-SR Official PyTorch implementation of our CVPR2021 paper MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Re

DV Lab 126 Dec 20, 2022
PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction... Attention in Attention Network for Image Super-Resolution (A2N) This repository is an PyTorch implementation of the paper "Atten

Haoyu Chen 71 Dec 30, 2022
PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

null 11 May 19, 2022