Image Deblurring using Generative Adversarial Networks

Overview

DeblurGAN

arXiv Paper Version

Pytorch implementation of the paper DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks.

Our network takes blurry image as an input and procude the corresponding sharp estimate, as in the example:

The model we use is Conditional Wasserstein GAN with Gradient Penalty + Perceptual loss based on VGG-19 activations. Such architecture also gives good results on other image-to-image translation problems (super resolution, colorization, inpainting, dehazing etc.)

How to run

Prerequisites

  • NVIDIA GPU + CUDA CuDNN (CPU untested, feedback appreciated)
  • Pytorch

Download weights from Google Drive . Note that during the inference you need to keep only Generator weights.

Put the weights into

/.checkpoints/experiment_name

To test a model put your blurry images into a folder and run:

python test.py --dataroot /.path_to_your_data --model test --dataset_mode single --learn_residual

Data

Download dataset for Object Detection benchmark from Google Drive

Train

If you want to train the model on your data run the following command to create image pairs:

python datasets/combine_A_and_B.py --fold_A /path/to/data/A --fold_B /path/to/data/B --fold_AB /path/to/data

And then the following command to train the model

python train.py --dataroot /.path_to_your_data --learn_residual --resize_or_crop crop --fineSize CROP_SIZE (we used 256)

Other Implementations

Keras Blog

Keras Repository

Citation

If you find our code helpful in your research or work please cite our paper.

@article{DeblurGAN,
  title = {DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks},
  author = {Kupyn, Orest and Budzan, Volodymyr and Mykhailych, Mykola and Mishkin, Dmytro and Matas, Jiri},
  journal = {ArXiv e-prints},
  eprint = {1711.07064},
  year = 2017
}

Acknowledgments

Code borrows heavily from pix2pix. The images were taken from GoPRO test dataset - DeepDeblur

Comments
  • Error message when I tried to run combine_A_and_B.py before training

    Error message when I tried to run combine_A_and_B.py before training

    My command in terminal is: python datasets/combine_A_and_B.py --fold_A ./train --fold_B ./val --fold_AB ./results and there is error message shown below [fold_A] = ./train [fold_B] = ./val [fold_AB] = ./results [num_imgs] = 1000000 [use_AB] = False Traceback (most recent call last): File "datasets/combine_A_and_B.py", line 23, in img_list = os.listdir(img_fold_A) NotADirectoryError: [Errno 20] Not a directory: './train/2163.jpg'

    Can anyone please help m with this? Thank you.

    opened by csqyin 11
  • TypeError: cuda() got an unexpected keyword argument 'device_id'

    TypeError: cuda() got an unexpected keyword argument 'device_id'

    When I run train.py in command line,I meet this problem. What shold I do?

    (E:\programs\Anaconda\envs\python) D:\cv2\DeblurGAN-master>python train.py --dataroot blurred_sharp/ --continue_train ------------ Options ------------- batchSize: 1 beta1: 0.5 checkpoints_dir: ./checkpoints continue_train: True dataroot: blurred_sharp/ dataset_mode: aligned display_freq: 100 display_id: 1 display_port: 8097 display_single_pane_ncols: 0 display_winsize: 256 epoch_count: 1 fineSize: 256 gan_type: wgan-gp gpu_ids: [0] identity: 0.0 input_nc: 3 isTrain: True lambda_A: 100.0 lambda_B: 10.0 learn_residual: False loadSizeX: 640 loadSizeY: 360 lr: 0.0001 max_dataset_size: inf model: content_gan nThreads: 2 n_layers_D: 3 name: experiment_name ndf: 64 ngf: 64 niter: 150 niter_decay: 150 no_dropout: False no_flip: False no_html: False norm: instance output_nc: 3 phase: train pool_size: 50 print_freq: 100 resize_or_crop: resize_and_crop save_epoch_freq: 5 save_latest_freq: 5000 serial_batches: False which_direction: AtoB which_epoch: latest which_model_netD: basic which_model_netG: resnet_9blocks -------------- End ---------------- CustomDatasetDataLoader dataset [AlignedDataset] was created Traceback (most recent call last): File "train.py", line 56, in model = create_model(opt) File "D:\cv2\DeblurGAN-master\models\models.py", line 11, in create_model model.initialize(opt) File "D:\cv2\DeblurGAN-master\models\conditional_gan_model.py", line 34, in initialize opt.which_model_netG, opt.norm, not opt.no_dropout, self.gpu_ids, use_parallel, opt.learn_residual) File "D:\cv2\DeblurGAN-master\models\networks.py", line 52, in define_G netG.cuda(device_id=gpu_ids[0]) TypeError: cuda() got an unexpected keyword argument 'device_id'

    opened by ilei00 10
  • The grid effect exists in the generate images. How to fix it?

    The grid effect exists in the generate images. How to fix it?

    image

    Just like the above image, the grid effect exists in the generate images, which make the generate images seem to be not natural. Would you meet this situation, and how to fix it?

    Thank you.

    opened by shenqixiaojiang 9
  •  No module named 'dominate'

    No module named 'dominate'

    I meet the error util/html.py", line 1, in import dominate ModuleNotFoundError: No module named 'dominate'

    By the way, if I want to run it on CPU,where I need to change?

    Thank you.

    opened by WuJunde 8
  • Backward twice when using wgan-gp as critic

    Backward twice when using wgan-gp as critic

    I encountered the following error when using wgan-gp as discriminator and everything went well whe using other discriminator: Traceback (most recent call last): File "train.py", line 42, in model.optimize_parameters() File "/home/gasoon/code/research/VideoDeblur/models/conditional_gan_model.py", line 136, in optimize_parameters self.backward_D() File "/home/gasoon/code/research/VideoDeblur/models/conditional_gan_model.py", line 110, in backward_D self.loss_D.backward() File "/home/gasoon/anaconda3/envs/vd/lib/python3.6/site-packages/torch/tensor.py", line 93, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/gasoon/anaconda3/envs/vd/lib/python3.6/site-packages/torch/autograd/init.py", line 89, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

    I'm using the data generated by myself whose resolution is 640*480. Does anyone know the way to solve it? Thank you so much!

    opened by Gasoonjia 7
  • How to get deblurred images of full resolution in test?

    How to get deblurred images of full resolution in test?

    Hello, I test the model on my images (1333x764) using following command , but I get 256x256 deblurred images, how can I get deblurred images of original resolution ?

    python test.py --dataroot /.path_to_your_data --model test --dataset_mode single --learn_residual

    It seems that the model cropped the image ,but i dont know how to avoid it.

    opened by Deephome 7
  • ImportError: cannot import name 'SSIM'

    ImportError: cannot import name 'SSIM'

    Hello! When I run train.py in command line,I meet this problem. What shold I do?

    python test.py --dataroot ./test --model test --dataset_mode single --learn_residual

    Traceback (most recent call last): File "test.py", line 10, in from ssim import SSIM ImportError: cannot import name 'SSIM'

    opened by hust-lidelong 6
  • ConnectionError:HTTPConnectionPool(host='localhost', port=8097)

    ConnectionError:HTTPConnectionPool(host='localhost', port=8097)

    When I train DeblurGAN,I received this error ,while test no this error. ConnectionError: HTTPConnectionPool(host='localhost', port=8097): Max retries exceeded with url: /events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f32c9ce9950>: Failed to establish a new connection: [Errno 111] Connection refused',)) How to solve this problem?

    opened by hrbspider 6
  • use_parallel is an undefined name in test_model.py

    use_parallel is an undefined name in test_model.py

    https://github.com/KupynOrest/DeblurGAN/search?utf8=%E2%9C%93&q=use_parallel&type=

    https://github.com/KupynOrest/DeblurGAN/blob/master/models/test_model.py#L18

    flake8 testing of https://github.com/KupynOrest/DeblurGAN on Python 2.7.14

    $ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

    ./models/test_model.py:18:105: F821 undefined name 'use_parallel'
                                          opt.which_model_netG, opt.norm, not opt.no_dropout, self.gpu_ids, use_parallel,
                                                                                                            ^
    
    opened by cclauss 6
  • On the detailed operation

    On the detailed operation

    Hello, I am a beginner. How can I add dataroot to the environment, and I want to turn my blurred picture into clear image. Which folder should I put into it and and what commands to input to Python?

    The following is my mistake [-h] --dataroot DATAROOT [--batchSize BATCHSIZE] [--loadSizeX LOADSIZEX] [--loadSizeY LOADSIZEY] [--fineSize FINESIZE] [--input_nc INPUT_NC] [--output_nc OUTPUT_NC] [--ngf NGF] [--ndf NDF] [--which_model_netD WHICH_MODEL_NETD] [--which_model_netG WHICH_MODEL_NETG] [--learn_residual] [--gan_type GAN_TYPE] [--n_layers_D N_LAYERS_D] [--gpu_ids GPU_IDS] [--name NAME] [--dataset_mode DATASET_MODE] [--model MODEL] [--which_direction WHICH_DIRECTION] [--nThreads NTHREADS] [--checkpoints_dir CHECKPOINTS_DIR] [--norm NORM] [--serial_batches] [--display_winsize DISPLAY_WINSIZE] [--display_id DISPLAY_ID] [--display_port DISPLAY_PORT] [--display_single_pane_ncols DISPLAY_SINGLE_PANE_NCOLS] [--no_dropout] [--max_dataset_size MAX_DATASET_SIZE] [--resize_or_crop RESIZE_OR_CROP] [--no_flip] [--ntest NTEST] [--results_dir RESULTS_DIR] [--aspect_ratio ASPECT_RATIO] [--phase PHASE] [--which_epoch WHICH_EPOCH] [--how_many HOW_MANY] test.py: error: the following arguments are required: --dataroot

    opened by CAAlexChang 5
  • Can someone please upload a requirements file to run the code?

    Can someone please upload a requirements file to run the code?

    I am trying to run the test code but I keep getting some dependency errors which I am pretty sure is happening because I got the wrong version of some package. It would be great if you could upload the requirements text file. Thanks a lot! @KupynOrest

    opened by prash030 4
  • Log Files from Training

    Log Files from Training

    Thank you for your awesome code!

    I am hoping you might open-source the log files you have from training. Maybe the training and validation loss as a function of epoch (and/or batch) with an estimate of the runtime?

    opened by gauenk 0
  • macOS thread error and timeout using CPU—Warning: Cannot set number of intraop threads after parallel work has started

    macOS thread error and timeout using CPU—Warning: Cannot set number of intraop threads after parallel work has started

    Testing out DeblurGAN on macOS Big Sur using anaconda and python 3.7.

    I got the error using the basic test command or a command with more flags with flag set for CPU:

    ## either command will timeout
    python test.py --dataroot blur_test --model test --dataset_mode single --gpu_ids -1
    
    python test.py --dataroot blur_test --model test --dataset_mode single --gpu_ids -1 --resize_or_crop scale_width --loadSizeX 720 --loadSizeY 960 --fineSize 720
    

    I tried setting --nThreads 1 but got the same error. However, I was able to bypass this by adding an environment variable to the command:

    ### both of these commands work
    OMP_NUM_THREADS=1 python test.py --dataroot blur_test --model test --dataset_mode single --gpu_ids -1
    
    OMP_NUM_THREADS=1 python test.py --dataroot blur_test --model test --dataset_mode single --gpu_ids -1 --resize_or_crop scale_width --loadSizeX 720 --loadSizeY 960 --fineSize 720
    

    Any way to avoid this error?

    Here is the full error and traceback:

    model [TestModel] was created
    [2021-11-11 15:22:47,942] WARNING: Setting up a new session...
    [2021-11-11 15:22:47,961] INFO: Visdom successfully connected to server
    [W ParallelNative.cpp:214] Warning: Cannot set number of intraop threads after parallel work has started or after set_num_threads call when using native parallel backend (function set_num_threads)
    [2021-11-11 15:22:48,107] DEBUG: STREAM b'IHDR' 16 13
    [2021-11-11 15:22:48,108] DEBUG: STREAM b'pHYs' 41 9
    [2021-11-11 15:22:48,109] DEBUG: STREAM b'IDAT' 62 380148
    ^CTraceback (most recent call last):
      File "test.py", line 31, in <module>
        for i, data in enumerate(dataset):
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
        data = self._next_data()
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data
        idx, data = self._get_data()
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1152, in _get_data
        success, data = self._try_get_data()
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 990, in _try_get_data
        data = self._data_queue.get(timeout=timeout)
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/multiprocessing/queues.py", line 104, in get
        if not self._poll(timeout):
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/multiprocessing/connection.py", line 257, in poll
        return self._poll(timeout)
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/multiprocessing/connection.py", line 414, in _poll
        r = wait([self], timeout)
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/multiprocessing/connection.py", line 921, in wait
        ready = selector.select(timeout)
      File "/opt/anaconda3/envs/DEBLURGAN/lib/python3.7/selectors.py", line 415, in select
        fd_event_list = self._selector.poll(timeout)
    KeyboardInterrupt
    
    opened by KeygenLLC 0
Owner
Orest Kupyn
Research Engineer from Lviv, Ukraine, mostly working in Deep Learning & Computer Vision
Orest Kupyn
PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

null 11 May 19, 2022
[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

KPAC: Kernel-Sharing Parallel Atrous Convolutional block This repository contains the official Tensorflow implementation of the following paper: Singl

Hyeongseok Son 50 Dec 29, 2022
StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

null 3k Jan 8, 2023
[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

Shengyu Zhao 373 Jan 2, 2023
π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis Project Page | Paper | Data Eric Ryan Chan*, Marco Monteiro*, Pe

null 375 Dec 31, 2022
Pytorch implementation for reproducing StackGAN_v2 results in the paper StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

StackGAN-v2 StackGAN-v1: Tensorflow implementation StackGAN-v1: Pytorch implementation Inception score evaluation Pytorch implementation for reproduci

Han Zhang 809 Dec 16, 2022
Flickr-Faces-HQ (FFHQ) is a high-quality image dataset of human faces, originally created as a benchmark for generative adversarial networks (GAN)

Flickr-Faces-HQ Dataset (FFHQ) Flickr-Faces-HQ (FFHQ) is a high-quality image dataset of human faces, originally created as a benchmark for generative

NVIDIA Research Projects 2.9k Dec 28, 2022
Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

SSRL-for-image-classification Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

Feng 2 Nov 19, 2021
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis Multi-View Consistent Generative Adversarial Networks for 3D-aware

Xuanmeng Zhang 78 Dec 10, 2022
SimDeblur is a simple framework for image and video deblurring, implemented by PyTorch

SimDeblur (Simple Deblurring) is an open source framework for image and video deblurring toolbox based on PyTorch, which contains most deep-learning based state-of-the-art deblurring algorithms. It is easy to implement your own image or video deblurring or other restoration algorithms.

null 220 Jan 7, 2023
Exploring Image Deblurring via Blur Kernel Space (CVPR'21)

Exploring Image Deblurring via Encoded Blur Kernel Space About the project We introduce a method to encode the blur operators of an arbitrary dataset

VinAI Research 118 Dec 19, 2022
A curated list of resources for Image and Video Deblurring

A curated list of resources for Image and Video Deblurring

Subeesh Vasu 1.7k Jan 1, 2023
[CVPR 2021] Official PyTorch Implementation for "Iterative Filter Adaptive Network for Single Image Defocus Deblurring"

IFAN: Iterative Filter Adaptive Network for Single Image Defocus Deblurring Checkout for the demo (GUI/Google Colab)! The GUI version might occasional

Junyong Lee 173 Dec 30, 2022
Code for the paper "TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks"

TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks This is a Python3 / Pytorch implementation of TadGAN paper. The associated

Arun 92 Dec 3, 2022
Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

This repository is the official PyTorch implementation of Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

hippopmonkey 4 Dec 11, 2022
pytorch implementation for Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arXiv:1609.04802

PyTorch SRResNet Implementation of Paper: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"(https://arxiv.org/abs

Jiu XU 436 Jan 9, 2023
PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Networks".

pix2pix-pytorch PyTorch implementation of Image-to-Image Translation Using Conditional Adversarial Networks. Based on pix2pix by Phillip Isola et al.

mrzhu 383 Dec 17, 2022
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Thomas Neumann 117 Nov 27, 2022
Regularizing Generative Adversarial Networks under Limited Data (CVPR 2021)

Regularizing Generative Adversarial Networks under Limited Data [Project Page][Paper] Implementation for our GAN regularization method. The proposed r

Google 148 Nov 18, 2022