Code of Adverse Weather Image Translation with Asymmetric and Uncertainty aware GAN

Jeong-gi Kwak

Last update: Dec 26, 2022

Related tags

Deep Learning AU-GAN

Overview

Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)

Official Tensorflow implementation of Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)
Jeong-gi Kwak, Youngsaeng Jin, Yuanming Li, Dongsik Yoon, Donghyeon Kim and Hanseok Ko
British Machine Vision Conference (BMVC), 2021

Intro

Night → Day (BDD100K)

Rainy night → Day (Alderdey)

Architecture

Our generator has asymmetric structure for editing day→night and night→day. Please refer our paper for details

Envs

git clone https://github.com/jgkwak95/AU-GAN.git
cd AU-GAN

# Create virtual environment
conda create -y --name augan python=3.6.7
conda activate augan

conda install tensorflow-gpu==1.14.0   # Tensorflow 1.14
pip install --no-cache-dir -r requirements.txt

Preparing datasets

Night → Day
Berkeley DeepDrive dataset contains 100,000 high resolution images of the urban roads for autonomous driving.

Rainy night → Day
Alderley dataset consists of images of two domains, rainy night and daytime. It was collected while driving the same route in each weather environment.

Please download datasets and then construct them following ForkGAN

Training

# Alderley (256x256)
python main_uncer.py --dataset_dir alderley
                     --phase train
                     --experiment_name alderley_exp
                     --batch_size 8 
                     --load_size 286 
                     --fine_size 256 
                     --use_uncertainty True

# BDD100k (512x512)
python main_uncer.py --dataset_dir bdd100k 
                     --phase train
                     --experiment_name bdd_exp
                     --batch_size 4 
                     --load_size 572 
                     --fine_size 512 
                     --use_uncertainty True

Test

# Alderley (256x256)
python main_uncer.py --dataset_dir alderley
                     --phase test
                     --experiment_name alderley_exp
                     --batch_size 1 
                     --load_size 286 
                     --fine_size 256

# BDD100k (512x512)
python main_uncer.py --dataset_dir bdd100k
                     --phase test
                     --experiment_name bdd_exp
                     --batch_size 1 
                     --load_size 572 
                     --fine_size 512

Additional results

More results in paper and supplementary

Uncertainty map

Citation

If our code is helpful your research, please cite our paper:

@InProceedings{kwak_adverse_2021},
  author = {Kwak, Jeong-gi and Jin, Youngsaeng and Li, Yuanming and Yoon, Dongsik and Kim, Donghyeon and Ko, Hanseok},
  title = {Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN},
  booktitle = {British Conference of Computer Vision (BMVC)},
  month = {November},
  year = {2021}
}

Acknowledgments

Our code is bulided upon the ForkGAN implementation.

π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis Project Page | Paper | Data Eric Ryan Chan*, Marco Monteiro*, Pe

375 Dec 31, 2022

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time Introduction This is official implementation for DR-GAN (IEEE TCS

18 Dec 23, 2022

This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

RGB2NIR_Experimental This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models

5 Jan 4, 2023

Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)

Aerial Depth Completion This work is described in the letter "Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation", by Lucas

70 Dec 22, 2022

Code for Dual Contrastive Learning for Unsupervised Image-to-Image Translation, NTIRE, CVPRW 2021.

arXiv Dual Contrastive Learning Adversarial Generative Networks (DCLGAN) We provide our PyTorch implementation of DCLGAN, which is a simple yet powerf

119 Dec 4, 2022

GAN Image Generator and Characterwise Image Recognizer with python

MODEL SUMMARY 모델의 구조는 크게 6단계로 나뉩니다. STEP 0: Input Image Predict 할 이미지를 모델에 입력합니다. STEP 1: Make Black and White Image STEP 1 은 입력받은 이미지의 글자를 흑색으로, 배경을

1 Feb 9, 2022

Code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty

Deep Deterministic Uncertainty This repository contains the code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic

69 Nov 28, 2022

AOT-GAN for High-Resolution Image Inpainting (codebase for image inpainting)

AOT-GAN for High-Resolution Image Inpainting Arxiv Paper | AOT-GAN: Aggregated Contextual Transformations for High-Resolution Image Inpainting Yanhong

214 Jan 3, 2023

the code for paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration"

EOW-Softmax This code is for the paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration". Accepted by ICCV21. Usage Commnd exa

36 Dec 2, 2022

Comments

Some strange results with the pre-trained file and BDD10K dataset.

Thanks for your great work! I tried to reproduce your work. But I found some strange results with the pre-trained file and BDD10K dataset. @jgkwak95

When the fine_size is set to 512, I got the black results:

When the fine_size is set to 1024, I got the scary results:

opened by hellohaozheng 10
Pretrained Model for testing on a single image.

Thankyou for your amazing work. I was interested in the pre-trained model. Is there a pre-trained model available to convert a single image from night -> day. Thankyou for your time. Looking forward for your response.

opened by Rodrigues-Royston 5
Alderley dataset

Firstly, thank you for sharing your wonderful work. I was about to try your project, but seems there is an unknown issue to the Alderley dataset. It is unavailable to access to the link. Also, I've tried to reach to the website via Google but seems the website itself has issues. If possible, could you provide a link(e.g. Google drive) to access to the Alderley dataset you have? Thank you.

opened by ejhung 2
Bump pillow from 6.0.0 to 8.3.2
Bumps pillow from 6.0.0 to 8.3.2.

Release notes

Sourced from pillow's releases.

8.3.2

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.2.html

Security

CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

Fix 6-byte OOB read in FliDecode [wiredfool]

Python 3.10 wheels

Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

Fixed regressions

Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

Updates for ImagePalette channel order #5599 [radarhere]

Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

8.3.1

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.1.html

Changes

Catch OSError when checking if fp is sys.stdout #5585 [@radarhere]

Handle removing orientation from alternate types of EXIF data #5584 [@radarhere]

Make Image.array take optional dtype argument #5572 [@t-vi]

8.3.0

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html

Changes

Use snprintf instead of sprintf #5567 [@radarhere]

Limit TIFF strip size when saving with LibTIFF #5514 [@kmilos]

Allow ICNS save on all operating systems #4526 [@newpanjing]

De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [@gofr]

Do not use background or transparency index for new color #5564 [@radarhere]

Simplified code #5315 [@radarhere]

Replaced xml.etree.ElementTree #5565 [@radarhere]

... (truncated)

Changelog

Sourced from pillow's changelog.

8.3.2 (2021-09-02)

CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

Fix 6-byte OOB read in FliDecode [wiredfool]

Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

Updates for ImagePalette channel order #5599 [radarhere]

Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

8.3.1 (2021-07-06)

Catch OSError when checking if fp is sys.stdout #5585 [radarhere]

Handle removing orientation from alternate types of EXIF data #5584 [radarhere]

Make Image.array take optional dtype argument #5572 [t-vi, radarhere]

8.3.0 (2021-07-01)

Use snprintf instead of sprintf. CVE-2021-34552 #5567 [radarhere]

Limit TIFF strip size when saving with LibTIFF #5514 [kmilos]

Allow ICNS save on all operating systems #4526 [baletu, radarhere, newpanjing, hugovk]

De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [gofr, radarhere]

Replaced xml.etree.ElementTree #5565 [radarhere]

... (truncated)

Commits

8013f13 8.3.2 version bump

23c7ca8 Update CHANGES.rst

8450366 Update release notes

a0afe89 Update test case

9e08eb8 Raise ValueError if color specifier is too long

bd5cf7d FLI tests for Oss-fuzz crash.

94a0cf1 Fix 6-byte OOB read in FliDecode

cece64f Add 8.3.2 (2021-09-02) [CI skip]

e422386 Add release notes for Pillow 8.3.2

08dcbb8 Pillow 8.3.2 supports Python 3.10 [ci skip]

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1

Owner

Jeong-gi Kwak

GitHub

Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

LiDAR fog simulation Created by Martin Hahner at the Computer Vision Lab of ETH Zurich. This is the official code release of the paper Fog Simulation

110 Dec 30, 2022

FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

FuseDream This repo contains code for our paper (paper link): FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizat

191 Dec 31, 2022

Code of Adverse Weather Image Translation with Asymmetric and Uncertainty aware GAN

Related tags

Overview

Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)

Intro

Night → Day (BDD100K)

Rainy night → Day (Alderdey)

Architecture

Envs

Preparing datasets

Training

Test

Additional results

Uncertainty map

Citation

Acknowledgments

You might also like...

π-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)

Code for Dual Contrastive Learning for Unsupervised Image-to-Image Translation, NTIRE, CVPRW 2021.

GAN Image Generator and Characterwise Image Recognizer with python

Code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty

AOT-GAN for High-Resolution Image Inpainting (codebase for image inpainting)

the code for paper "Energy-Based Open-World Uncertainty Modeling for Confidence Calibration"

Comments

Some strange results with the pre-trained file and BDD10K dataset.

Pretrained Model for testing on a single image.

Alderley dataset

Bump pillow from 6.0.0 to 8.3.2

8.3.2

Security

Python 3.10 wheels

Fixed regressions

8.3.1

Changes

8.3.0

Changes

8.3.2 (2021-09-02)

8.3.1 (2021-07-06)

8.3.0 (2021-07-01)

Owner

Jeong-gi Kwak

Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

Speech Enhancement Generative Adversarial Network Based on Asymmetric AutoEncoder

(SIGIR2020) “Asymmetric Tri-training for Debiasing Missing-Not-At-Random Explicit Feedback’’

Asymmetric Bilateral Motion Estimation for Video Frame Interpolation, ICCV2021

PyTorch implementation of Asymmetric Siamese (https://arxiv.org/abs/2204.00613)

Symmetry and Uncertainty-Aware Object SLAM for 6DoF Object Pose Estimation

Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving

Official PyTorch implementation of UACANet: Uncertainty Aware Context Attention for Polyp Segmentation

Text to Image Generation with Semantic-Spatial Aware GAN