PyTorch implementation of 1712.06087 "Zero-Shot" Super-Resolution using Deep Internal Learning

Overview

Unofficial PyTorch implementation of "Zero-Shot" Super-Resolution using Deep Internal Learning

Unofficial Implementation of 1712.06087 "Zero-Shot" Super-Resolution using Deep Internal Learning by Assaf Shocher, Nadav Cohen, Michal Irani.

Official Project page: http://www.wisdom.weizmann.ac.il/~vision/zssr/

Paper: https://arxiv.org/abs/1712.06087


This trains a deep neural network to perform super resolution using a single image.

The network is not trained on additional images, and only uses information from within the target image. Pairs of high resolution and low resolution patches are sampled from the image, and the network fits their difference.

Low resolution ZSSR

ZSSR ZSSR


TODO:

  • Implement additional augmentation using the "Geometric self ensemble" mentioned in the paper.
  • Implement gradual increase of the super resolution factor as described in the paper.
  • Support for arbitrary kernel estimation and sampling with arbitrary kernels. The current implementation interpolates the images bicubic interpolation.

Deviations from paper:

  • Instead of fitting the loss and analyzing it's standard deviation, the network is trained for a constant number of batches. The learning rate shrinks x10 every 10,000 iterations.

Usage

Example: python train.py --img img.png

usage: train.py [-h] [--num_batches NUM_BATCHES] [--crop CROP] [--lr LR]
                [--factor FACTOR] [--img IMG]

optional arguments:
  -h, --help            show this help message and exit
  --num_batches NUM_BATCHES
                        Number of batches to run
  --crop CROP           Random crop size
  --lr LR               Base learning rate for Adam
  --factor FACTOR       Interpolation factor.
  --img IMG             Path to input img
Comments
  • FileNotFoundError: [Errno 2] No such file or directory: 'lincoln.png'

    FileNotFoundError: [Errno 2] No such file or directory: 'lincoln.png'

    (torch14) C:\Users\15479\Desktop\pytorch-zssr-master>python train.py --img lincoln.png Traceback (most recent call last): File "train.py", line 105, in img = PIL.Image.open(args.img) File "E:\anaconda\envs\torch14\lib\site-packages\PIL\Image.py", line 2891, in open fp = builtins.open(filename, "rb") FileNotFoundError: [Errno 2] No such file or directory: 'lincoln.png'

    I'm a green hand.Please tell me What should be after --img? How to set? Please ,help me bro!!!

    opened by ymtupup 1
  • IndexError: too many indices for array on line 46 in train, cpu_loss = error.data.cpu().numpy()[0]

    IndexError: too many indices for array on line 46 in train, cpu_loss = error.data.cpu().numpy()[0]

    (base) C:\Users\Luke Chen\Desktop\pytorch-zssr-master>python train.py --crop 30 --num_batches 8 --img img.png
    train.py:18: UserWarning: nn.init.kaiming_normal is now deprecated in favor of nn.init.kaiming_normal_.
      init.kaiming_normal(m.weight.data, a=0, mode='fan_in')
      0%|                                                                                            | 0/8 [00:00<?, ?it/s]
    Traceback (most recent call last):
      File "train.py", line 118, in <module>
        train(model, img, args.factor, args.num_batches, args.lr, args.crop)
      File "train.py", line 46, in train
        cpu_loss = error.data.cpu().numpy()[0]
    IndexError: too many indices for array
    
    opened by inkplay 1
  • Code looks very different from what was proposed in the original repository

    Code looks very different from what was proposed in the original repository

    Hi, so I compared the code (overall skeletal structure) with the original repository that was written in TensorFlow and a lot of concepts like kernel implementation, hr-father, lr-sons etc.. weren't found in this code.

    Can you help me understand if I have missed something?

    opened by IamGroot19 0
  • compare with the original txt

    compare with the original txt

    I use the defalut parameters. lr = 1e--5 nums = 15000 in fact, I test in Set14,and get the results as: x2,psnr:29.75,ssim:0.88 however, the original txt is: x2,psnr:33.00,ssim:0.9108. I want to know ,did you test in Set14,and what's your results.

    opened by riskeverything 0
  • CUDA out of memory.

    CUDA out of memory.

    I ran everything with default settings just to test the code ( run train.py --img lincoln.png). I got the 2x enlarged image.

    Then I used this 2x image as input to magnify it to 4x. During this run, all the 15000 epochs were completed and a new image file called 'low_res.png' was found in the folder. But after this, the program stopped without completion with the following message :

    CUDA out of memory. Tried to allocate 788.00 MiB (GPU 0; 4.00 GiB total capacity; 2.38 GiB already allocated; 531.39 MiB free; 128.68 MiB cached)

    Also, during the running of the program, I checked the task manager and it was always like CPU(near 100%), GPU(2%).

    Can someone help me understand the issue and help me find a solution?

    opened by IamGroot19 0
  • Kernel estimation?

    Kernel estimation?

    I was wondering if you had a chance to look into the kernel estimation part of the paper, as that is what really lets ZSSR be more useful than other SR methods (per the paper). If not, do you have any guidance for trying to implement it? In particular, how can image resizing with an arbitrary kernel be done?

    opened by sachit-menon 0
  • Poor inference result

    Poor inference result

    I set "lincoln.png" in the example folder as input and run the code. It turns out that the result is hardly different from the input LR. I took a brief look at the code but nothing seems wrong. I wonder if you had the same problem.

    opened by jaegerstar 7
Owner
Jacob Gildenblat
Machine learning / Computer Vision.
Jacob Gildenblat
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1.2k Dec 26, 2022
Pytorch implementation of Deep Recursive Residual Network for Super Resolution (DRRN)

DRRN-pytorch This is an unofficial implementation of "Deep Recursive Residual Network for Super Resolution (DRRN)", CVPR 2017 in Pytorch. [Paper] You

yun_yang 192 Dec 12, 2022
Official PyTorch implementation of the paper "Deep Constrained Least Squares for Blind Image Super-Resolution", CVPR 2022.

Deep Constrained Least Squares for Blind Image Super-Resolution [Paper] This is the official implementation of 'Deep Constrained Least Squares for Bli

MEGVII Research 141 Dec 30, 2022
Official Pytorch Implementation of: "Semantic Diversity Learning for Zero-Shot Multi-label Classification"(2021) paper

Semantic Diversity Learning for Zero-Shot Multi-label Classification Paper Official PyTorch Implementation Avi Ben-Cohen, Nadav Zamir, Emanuel Ben Bar

null 28 Aug 29, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page >> coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page >> coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022
Official Pytorch implementation of Meta Internal Learning

Official Pytorch implementation of Meta Internal Learning

null 10 Aug 24, 2022
Implementation of paper: "Image Super-Resolution Using Dense Skip Connections" in PyTorch

SRDenseNet-pytorch Implementation of paper: "Image Super-Resolution Using Dense Skip Connections" in PyTorch (http://openaccess.thecvf.com/content_ICC

wxy 114 Nov 26, 2022
pytorch implementation for Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arXiv:1609.04802

PyTorch SRResNet Implementation of Paper: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"(https://arxiv.org/abs

Jiu XU 436 Jan 9, 2023
[CVPR 2022] Official PyTorch Implementation for "Reference-based Video Super-Resolution Using Multi-Camera Video Triplets"

Reference-based Video Super-Resolution (RefVSR) Official PyTorch Implementation of the CVPR 2022 Paper Project | arXiv | RealMCVSR Dataset This repo c

Junyong Lee 151 Dec 30, 2022
PyTorch code for 'Efficient Single Image Super-Resolution Using Dual Path Connections with Multiple Scale Learning'

Efficient Single Image Super-Resolution Using Dual Path Connections with Multiple Scale Learning This repository is for EMSRDPN introduced in the foll

null 7 Feb 10, 2022
PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

About PyTorch 1.2.0 Now the master branch supports PyTorch 1.2.0 by default. Due to the serious version problem (especially torch.utils.data.dataloade

Sanghyun Son 2.1k Jan 1, 2023
An official implementation of "Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation" (ICCV 2021) in PyTorch.

Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation This is an official implementation of the paper "Exploiting a Joint

CV Lab @ Yonsei University 35 Oct 26, 2022
GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks

GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks This repository implements a capsule model Inten

Joel Huang 15 Dec 24, 2022
Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic [Paper] [Colab is coming soon] Approach Example Usage To r

null 6 Dec 1, 2021
Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

kongdebug 14 Oct 14, 2022
Official implementation of Deep Burst Super-Resolution

Deep-Burst-SR Official implementation of Deep Burst Super-Resolution Publication: Deep Burst Super-Resolution. Goutam Bhat, Martin Danelljan, Luc Van

Goutam Bhat 113 Dec 19, 2022
Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution"

NTIRE2017 Super-resolution Challenge: SNU_CVLab Introduction This is our project repository for CVPR 2017 Workshop (2nd NTIRE). We, Team SNU_CVLab, (B

Bee Lim 625 Dec 30, 2022
Official implementation of Unfolded Deep Kernel Estimation for Blind Image Super-resolution.

Unfolded Deep Kernel Estimation for Blind Image Super-resolution Hongyi Zheng, Hongwei Yong, Lei Zhang, "Unfolded Deep Kernel Estimation for Blind Ima

Z80 15 Dec 26, 2022