Image De-raining Using a Conditional Generative Adversarial Network

Related tags

Deep Learning deep-learning gan id-cgan rain-removal
Overview

Image De-raining Using a Conditional Generative Adversarial Network

[Paper Link]

[Project Page]

He Zhang, Vishwanath Sindagi, Vishal M. Patel

In this paper, we investigate a new point of view in addressing single image de-raining problem. Instead of focusing only on deciding what is a good prior or a good framework to achieve good quantitative and qualitative performance, we also ensure that the de-rained image does not degrade the performance of a given computer vision algorithm such as detection and classification. In other words, the de-rained result should be indistinguishable from its corresponding clear image to a given discriminator. This criterion can be directly incorporated into the optimization framework by using the recently introduced conditional generative adversarial networks (GANs). To minimize artifacts introduced by GANs and ensure better visual quality, a new refined loss function is introduced.

@article{zhang2017image,		
  title={Image De-raining Using a Conditional Generative Adversarial Network},
  author={Zhang, He and Sindagi, Vishwanath and Patel, Vishal M},
  journal={arXiv preprint arXiv:1701.05957},
  year={2017}
}

Prepare

Instal torch7

Install nngraph

Install hdf5

Download the dataset from (https://drive.google.com/drive/folders/0Bw2e6Q0nQQvGbi1xV1Yxd09rY2s?resourcekey=0-dUoT9AJl1q6fXow9t5TcRQ&usp=sharing) and put the dataset folder into the "IDCGAN" folder

Training

DATA_ROOT=./datasets/rain name=rain which_direction=BtoA th train.lua

Testing

DATA_ROOT=./datasets/rain name=rain which_direction=BtoA phase=test_nature th test.lua

Testing using ID-CGAN model

The trained ID-CGAN model and our training and testing datasets can be found at (https://drive.google.com/drive/folders/0Bw2e6Q0nQQvGbi1xV1Yxd09rY2s?resourcekey=0-dUoT9AJl1q6fXow9t5TcRQ&usp=sharing)

*Make sure you download the vgg model that used for perceotual loss and put it in the ./IDCGAN/per_loss/models

##Acknowledgments##

Code borrows heavily from [pix2pix] and [Perceptual Loss]. Thanks for the sharing.

Comments
  • Quality measures in source code

    Quality measures in source code

    At first, thank you for your awesome source code and it ran perfectly like you said in the paper.

    I read your paper and source code, but can not found any places in the source code which were implemented and showed the quality measures as "Peak Signal to Noise Ratio (PSNR), Structural Similarity Index (SSIM) [48], Universal Quality Index (UQI) [49] and Visual Information Fidelity (VIF)[50]" in the paper.

    Could you please show me where and how did you evaluate these criteria. Thanks in advance.

    opened by TrinhQuocNguyen 6
  • Whether or not the code can process real time videos using opencv

    Whether or not the code can process real time videos using opencv

    Hello Hezhangsprinter, Thank you for your hard work. I have been played around with your code, and did some modify to process sequence images (image by image) using opencv, but when I try to process a real time video which is captured from camera, it seems to me that the process is slow. And after investigating, I have found that the code of converting the RGB <-> BGR consumes a lot of time. Do you have any ideas to improve it? Thank you.

    opened by TrinhQuocNguyen 3
  • the trained discriminator

    the trained discriminator

    Can you please also share your trained discriminator?

    I saw that in the configuration file, the number of iteration is set to be 2000. I'm wondering if that is necessary to achieve your performance.

    opened by xinario 2
  • quantitative measures

    quantitative measures

    Are you agree that PSNR, SSIM, UQI and VIF are reasonable quantitative metrics for synthetic images? Would you please release the evaluation implementation including the comparison with the baselines? Thanks a lot : )

    opened by lizhuorong 1
  • error

    error

    hi, I just follow the instructions you give, but I encounter the following error:

    @imcl-test:/ID-CGAN/IDCGAN$ DATA_ROOT=./datasets/rain name=rain which_direction=BtoA th train.lua

Found Environment variable CUDNN_PATH = /data/XXX/cuda/lib64/libcudnn.so.5.1.10	

/data/XXX/torch/install/bin/luajit: cannot open <per_loss/models/vgg16.t7> in mode r  at /data/XXX/torch/pkg/torch/lib/TH/THDiskFile.c:670

stack traceback:
	[C]: at 0x7f52554e1ad0
	[C]: in function 'DiskFile'
	/data/XXX/torch/install/share/lua/5.1/torch/File.lua:405: in function 'load'
	train.lua:68: in main chunk
	[C]: in function 'dofile'
	...eigq/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk
	[C]: at 0x00406670

    can you help me? @hezhangsprinter thanks

    opened by weigq 1
  • Latest version of the code

    Latest version of the code

    Does the publication from 2019, https://ieeexplore.ieee.org/document/8727938, use the same code as the one from 2017, https://arxiv.org/pdf/1701.05957.pdf? Could you upload the latest version of the code?

    Your results are impressive! We'd like to use it in our project for removing noise from data. We would appreciate your help!

    opened by chrismaliszewski 0
  • Dense Structure in the Implementation of the Generator Model

    Dense Structure in the Implementation of the Generator Model

    Sorry, I am new to Lua. But it seems the network model in the source code is not consistent with those defined in your paper. I did not recognize the DenseNet structure in your source code and it seems very similar to the original U-Net in the pixel2pixel model. Am I misunderstanding the code? Thank you so much for your attention.

    opened by KANG-Xudong 1
  • Cite article

    Cite article

    Hello, I am your reader. I want to cite your article, but the volume number, issue number and page number of your article cannot be found. Can you tell me this information if it is convenient for you? Thank you very much

    opened by FUNGERN 0
Owner
He Zhang
Research Scientist@Adobe, Phd in Computer Vision, Deep Learning
He Zhang
GitHub http://www.rci.rutgers.edu/~vmp93/index_ImageDeRaining.html
PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Networks".

pix2pix-pytorch PyTorch implementation of Image-to-Image Translation Using Conditional Adversarial Networks. Based on pix2pix by Phillip Isola et al.

mrzhu 383 Dec 17, 2022
Image-to-Image Translation with Conditional Adversarial Networks (Pix2pix) implementation in keras

pix2pix-keras Pix2pix implementation in keras. Original paper: Image-to-Image Translation with Conditional Adversarial Networks (pix2pix) Paper Author

William Falcon 141 Dec 30, 2022
pytorch implementation for Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network arXiv:1609.04802

PyTorch SRResNet Implementation of Paper: "Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network"(https://arxiv.org/abs

Jiu XU 436 Jan 9, 2023
A method that utilized Generative Adversarial Network (GAN) to interpret the black-box deep image classifier models by PyTorch.

A method that utilized Generative Adversarial Network (GAN) to interpret the black-box deep image classifier models by PyTorch.

Yunxia Zhao 3 Dec 29, 2022
Image Deblurring using Generative Adversarial Networks

DeblurGAN arXiv Paper Version Pytorch implementation of the paper DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks. Our netwo

Orest Kupyn 2.2k Jan 1, 2023
VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech Jaehyeon Kim, Jungil Kong, and Juhee Son In our rece

Jaehyeon Kim 1.7k Jan 8, 2023
Speech Enhancement Generative Adversarial Network Based on Asymmetric AutoEncoder

ASEGAN: Speech Enhancement Generative Adversarial Network Based on Asymmetric AutoEncoder 中文版简介 Readme with English Version 介绍 基于SEGAN模型的改进版本，使用自主设计的非

Nitin 53 Nov 17, 2022
House-GAN++: Generative Adversarial Layout Refinement Network towards Intelligent Computational Agent for Professional Architects

House-GAN++ Code and instructions for our paper: House-GAN++: Generative Adversarial Layout Refinement Network towards Intelligent Computational Agent

null 122 Dec 28, 2022
Python code to generate art with Generative Adversarial Network

GAN_Canvas_Maker Generating Art using Generative Adversarial Network (GAN) Python code to generate art with Generative Adversarial Network: https://to

Jonny Banana 10 Aug 22, 2022
Collection of TensorFlow2 implementations of Generative Adversarial Network varieties presented in research papers.

TensorFlow2-GAN Collection of tf2.0 implementations of Generative Adversarial Network varieties presented in research papers. Model architectures will

null 41 Apr 28, 2022
Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

?? Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) ?? If DaGAN is helpful in your photos/projects, please hel

Fa-Ting Hong 503 Jan 4, 2023
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021) This repository is the official P

Jingyun Liang 159 Dec 30, 2022
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021) This repository is the official P

Jingyun Liang 159 Dec 30, 2022
[ICLR 2021, Spotlight] Large Scale Image Completion via Co-Modulated Generative Adversarial Networks

Large Scale Image Completion via Co-Modulated Generative Adversarial Networks, ICLR 2021 (Spotlight) Demo | Paper [NEW!] Time to play with our interac

Shengyu Zhao 373 Jan 2, 2023
π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis Project Page | Paper | Data Eric Ryan Chan*, Marco Monteiro*, Pe

null 375 Dec 31, 2022
Pytorch implementation for reproducing StackGAN_v2 results in the paper StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

StackGAN-v2 StackGAN-v1: Tensorflow implementation StackGAN-v1: Pytorch implementation Inception score evaluation Pytorch implementation for reproduci

Han Zhang 809 Dec 16, 2022
Flickr-Faces-HQ (FFHQ) is a high-quality image dataset of human faces, originally created as a benchmark for generative adversarial networks (GAN)

Flickr-Faces-HQ Dataset (FFHQ) Flickr-Faces-HQ (FFHQ) is a high-quality image dataset of human faces, originally created as a benchmark for generative

NVIDIA Research Projects 2.9k Dec 28, 2022
Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

SSRL-for-image-classification Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

Feng 2 Nov 19, 2021
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis Multi-View Consistent Generative Adversarial Networks for 3D-aware

Xuanmeng Zhang 78 Dec 10, 2022