The code for 'Deep Residual Fourier Transformation for Single Image Deblurring'

Overview

PWC PWC PWC PWC PWC PWC

Deep Residual Fourier Transformation for Single Image Deblurring

Xintian Mao, Yiming Liu, Wei Shen, Qingli Li and Yan Wang

code will be released soon

Paper: https://arxiv.org/abs/2111.11745

Network Architecture

Overall Framework of DeepRFT

Installation

The model is built in PyTorch 1.8.0 and tested on Ubuntu 18.04 environment (Python3.8, CUDA11.1).

For installing, follow these intructions

conda create -n pytorch python=3.8
conda activate pytorch
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
pip install matplotlib scikit-image opencv-python yacs joblib natsort h5py tqdm kornia tensorboard ptflops

Install warmup scheduler

cd pytorch-gradual-warmup-lr; python setup.py install; cd ..

Quick Run

To test the pre-trained models of Deblur and Defocus Google Drive or 百度网盘 on your own images, run

python test.py --weights ckpt_path_here --input_dir path_to_images --result_dir save_images_here --win_size 256 # deblur
python test.py --weights ckpt_path_here --input_dir path_to_images --result_dir save_images_here --win_size 512 # defocus

Here is an example to train:

python train.py

Results

Experiment for image deblurring.

Deblurring on GoPro Datasets.

Reference Code:

Citation

If you use DeepRFT, please consider citing:

@inproceedings{,
    title={Deep Residual Fourier Transformation for Single Image Deblurring},
    author={Xintian Mao, Yiming Liu, Wei Shen, Qingli Li, Yan Wang},
    booktitle={arXiv:2111.11745},
    year={2021}
}

Contact

If you have any question, please contact [email protected]

Comments
  • MSFR loss

    MSFR loss

    Hi @INVOKERer. In MIMO-UNET, where pytorch version is 1.4, the FFT for MSFR loss is calculated as torch.rfft(image, signal_ndim=2, normalized=False, onesided=False)

    Could you, please, tell me how do you port it to torch.fft.rfft in pytorch 1.8?

    opened by misakshoyan 6
  • SSIM

    SSIM

    When you test SSIM ,did you use the function from skimage.metrics import structural_similarity?And set structural_similarity(p_numpy, label_numpy, data_range=1,multichannel=True)

    opened by EKChloe 4
  • Fast Fourier convolutions

    Fast Fourier convolutions

    Thanks for the good sharing. I concern about the ablation study on FFC (in Table 7). Did you directly replace the FT branch with FFC[41], which uses the BN operation (harmful to deblurring) ?

    opened by imageCompression1995 4
  • about the gopro dataset

    about the gopro dataset

    The gopro dataset provides two kinds of blurry images, i.e., 'blur' & 'blur_gamma'. And the original paper use 'blur_gamma' for experiments. But in your paper, it seems that you use the 'blur' dataset. I wonder if you have noticed this issue? And are all the competing methods trained on 'blur'?

    opened by dawnlh 2
  • "Res FFT-Conv Block" in MPRNet

    In MPRNet, the Channel Attention Block (CAB) has the following structure:

    `class CAB(nn.Module): def init(self, n_feat, kernel_size, reduction, bias, act): super(CAB, self).init() modules_body = []

        # ResBlock
        modules_body.append(conv(n_feat, n_feat, kernel_size, bias=bias))
        modules_body.append(act)
        modules_body.append(conv(n_feat, n_feat, kernel_size, bias=bias))
        ###
    
        self.CA = CALayer(n_feat, reduction, bias=bias)
        self.body = nn.Sequential(*modules_body)
    
    def forward(self, x):
        res = self.body(x)
        res = self.CA(res)
        res += x
        return res`
    

    In the paper, for Table 1, it is stated that ResBlock is replaced by "Res FFT-Conv Block". I wonder how do you integrate the "Res FFT-Conv Block" into CAB in your experiments? Do you just replace the ResBlock structure by "Res FFT-Conv Block" (please see my comments in the code) or you replace the whole CAB by "Res FFT-Conv Block"?

    Thanks in advance.

    opened by misakshoyan 1
  • Complete training loss curve

    Complete training loss curve

    Thanks for your wonderful deblurring work. I noticed that the provided loss curve in paper is only 120, and the epoch settting is 3000. Could you provide a complete training loss curve and complete performance curve on GoPro test dataset? Thank you.

    opened by suiyizhao 1
  • wraning

    wraning

    UserWarning: Casting complex values to real discards the imaginary part (Triggered internally at ../aten/src/ATen/native/Copy.cpp:239.) Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass 这是咋回事

    opened by shixiaoxueba 1
  • RuntimeError: cuFFT error: CUFFT_INVALID_SIZE

    RuntimeError: cuFFT error: CUFFT_INVALID_SIZE

    @INVOKERer 您好 作者 ,您的作品非常具有创造性,以至于我不得不加入我的网络尝试一下效果,我在训练的过程中遇到了相关的问题,您有遇到过如下的代码吗?

      0%|          | 0/10 [44:48<?, ?it/s]
    Traceback (most recent call last):
      File "E:\UIDGain++\train.py", line 101, in <module>
        main()
      File "E:\UIDGain++\train.py", line 59, in main
        model.update_D(images_a, images_b)
      File "E:\UIDGain++\model.py", line 180, in update_D
        self.forward()
      File "E:\UIDGain++\model.py", line 128, in forward
        self.z_content_i, self.z_content_b = self.enc_c.forward(self.real_I_encoded, self.real_B_encoded)
      File "E:\UIDGain++\networks.py", line 135, in forward
        outputA = self.convA(xa)
      File "E:\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
        return forward_call(*input, **kwargs)
      File "E:\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\container.py", line 139, in forward
        input = module(input)
      File "E:\anaconda3\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
        return forward_call(*input, **kwargs)
      File "E:\UIDGain++\networks.py", line 798, in forward
        y = torch.fft.rfft2(x, norm=self.norm)
    RuntimeError: cuFFT error: CUFFT_INVALID_SIZE
    
    Process finished with exit code 1
    
    opened by HelloWorldYYYYY 3
  • Log Files from Training

    Log Files from Training

    Thank you for your awesome code!

    I am hoping you might open-source the log files you have from training. Maybe the training and validation loss as a function of epoch (and/or batch) with an estimate of the runtime?

    opened by gauenk 1
Owner
null
Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution"

NTIRE2017 Super-resolution Challenge: SNU_CVLab Introduction This is our project repository for CVPR 2017 Workshop (2nd NTIRE). We, Team SNU_CVLab, (B

Bee Lim 625 Dec 30, 2022
[CVPR 2021] Official PyTorch Implementation for "Iterative Filter Adaptive Network for Single Image Defocus Deblurring"

IFAN: Iterative Filter Adaptive Network for Single Image Defocus Deblurring Checkout for the demo (GUI/Google Colab)! The GUI version might occasional

Junyong Lee 173 Dec 30, 2022
[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

KPAC: Kernel-Sharing Parallel Atrous Convolutional block This repository contains the official Tensorflow implementation of the following paper: Singl

Hyeongseok Son 50 Dec 29, 2022
PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

PyTorch code for our ECCV 2018 paper "Image Super-Resolution Using Very Deep Residual Channel Attention Networks"

Yulun Zhang 1.2k Dec 26, 2022
Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

This repository is the official PyTorch implementation of Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

hippopmonkey 4 Dec 11, 2022
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Kai Zhang 1.2k Dec 29, 2022
Image Super-Resolution Using Very Deep Residual Channel Attention Networks

Image Super-Resolution Using Very Deep Residual Channel Attention Networks

kongdebug 14 Oct 14, 2022
SimDeblur is a simple framework for image and video deblurring, implemented by PyTorch

SimDeblur (Simple Deblurring) is an open source framework for image and video deblurring toolbox based on PyTorch, which contains most deep-learning based state-of-the-art deblurring algorithms. It is easy to implement your own image or video deblurring or other restoration algorithms.

null 220 Jan 7, 2023
PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

null 11 May 19, 2022
Image Deblurring using Generative Adversarial Networks

DeblurGAN arXiv Paper Version Pytorch implementation of the paper DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks. Our netwo

Orest Kupyn 2.2k Jan 1, 2023
Exploring Image Deblurring via Blur Kernel Space (CVPR'21)

Exploring Image Deblurring via Encoded Blur Kernel Space About the project We introduce a method to encode the blur operators of an arbitrary dataset

VinAI Research 118 Dec 19, 2022
A curated list of resources for Image and Video Deblurring

A curated list of resources for Image and Video Deblurring

Subeesh Vasu 1.7k Jan 1, 2023
Apply a perspective transformation to a raster image inside Inkscape (no need to use an external software such as GIMP or Krita).

Raster Perspective Apply a perspective transformation to bitmap image using the selected path as envelope, without the need to use an external softwar

s.ouchene 19 Dec 22, 2022
Pytorch implementation of Deep Recursive Residual Network for Super Resolution (DRRN)

DRRN-pytorch This is an unofficial implementation of "Deep Recursive Residual Network for Super Resolution (DRRN)", CVPR 2017 in Pytorch. [Paper] You

yun_yang 192 Dec 12, 2022
A PyTorch implementation for PyramidNets (Deep Pyramidal Residual Networks)

A PyTorch implementation for PyramidNets (Deep Pyramidal Residual Networks) This repository contains a PyTorch implementation for the paper: Deep Pyra

Greg Dongyoon Han 262 Jan 3, 2023
Reproduce ResNet-v2(Identity Mappings in Deep Residual Networks) with MXNet

Reproduce ResNet-v2 using MXNet Requirements Install MXNet on a machine with CUDA GPU, and it's better also installed with cuDNN v5 Please fix the ran

Wei Wu 531 Dec 4, 2022
Resco: A simple python package that report the effect of deep residual learning

resco Description resco is a simple python package that report the effect of dee

Pierre-Arthur Claudé 1 Jun 28, 2022
DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021)

Evaluation, Training, Demo, and Inference of DeFMO DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021) Denys Rozumnyi, Martin R. O

Denys Rozumnyi 139 Dec 26, 2022
Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes (CVPR2021)

RSCD (BS-RSCD & JCD) Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes (CVPR2021) by Zhihang Zhong, Yinqiang Zheng, Imari Sato We co

null 81 Dec 15, 2022