This is an implementation for the CVPR2020 paper "Learning Invariant Representation for Unsupervised Image Restoration"

Overview

Learning Invariant Representation for Unsupervised Image Restoration (CVPR 2020)

Introduction

This is an implementation for the paper "Learning Invariant Representation for Unsupervised Image Restoration" (CVPR 2020), a simple and efficient framework for unsupervised image restoration, which is injected into the general domain transfer architecture. More details could be found in the original paper.

Network Architecture

test

Proposed method aims to learn the intermediate representation free of noise from corrupted input that $z_{x}$and align it with $z_{y}$ from clean image in the latent space $Z$. In addition, adversarial domain adaption and self-supervised constraints are introduced into our architecture. As shown in Fig1-(b), our method is more straight and effective than other domain-transfer methods, e.g., CycleGAN, UNIT, DRIT and so on.

Prerequisites

  • (OS) Windows/Ubuntu
  • Python >= 3.6
  • Pytorch >= 1.1.0
  • Python-Libs, e.g., cv2, skimage.

Training

  • Prepare your dataset. In our experiments, we used the PascalVoc dataset to generate training data for Gaussian noise removal.
  • Generate Gaussian or Poisson noise via skimage-lib.
  • Update the data paths in config.py and utils.py file.
  • Train your model by the train.py file.

Test

A simple script to test your model:

python3 test.py

Results

  • Gaussian Noise Removal

  • Poisson Noise Removal

  • Medical Image Denoising (Low-Dose CT)

Extending for other IR tasks

You could extend this work for other image restoration tasks, e.g., super-resolution, deblurring and so on. If so, you need to adjust some hyperparameters for them, and extra self-supervised modules also need to be altered. In this paper, we just provide a more general idea to process the unsupervised image restoration tasks via representation learning.

Acknowledge

Our code is based on the UNIT, which is a nice work for unsupervised image translation.

You might also like...
The project is an official implementation of our CVPR2019 paper
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Regularized Greedy Forest Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better r

Official implementation of AAAI-21 paper
Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

Exploring simple siamese representation learning This is a PyTorch re-implementation of the SimSiam paper on ImageNet dataset. The results match that

Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting.
Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting.

Non-AR Spatial-Temporal Transformer Introduction Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series For

This is a Pytorch implementation of the paper: Self-Supervised Graph Transformer on Large-Scale Molecular Data.

This is a Pytorch implementation of the paper: Self-Supervised Graph Transformer on Large-Scale Molecular Data.

Official implementation of the ICLR 2021 paper
Official implementation of the ICLR 2021 paper

You Only Need Adversarial Supervision for Semantic Image Synthesis Official PyTorch implementation of the ICLR 2021 paper "You Only Need Adversarial S

Implementation of Nyström Self-attention, from the paper Nyströmformer
Implementation of Nyström Self-attention, from the paper Nyströmformer

Nyström Attention Implementation of Nyström Self-attention, from the paper Nyströmformer. Yannic Kilcher video Install $ pip install nystrom-attention

Comments
  • Only one encoder in trainer.py

    Only one encoder in trainer.py

    Hi. The paper says there are two encoders (generators) for the clean image and noisy image splitly. However, I can only find one generator 'gen_a' in 'train.py'. I think that it is the generator for noisy image. Where is the defination of the other generator (for clean image)? Thansk in advance.

    opened by petruskkke 1
  • About the source and placement of the data set

    About the source and placement of the data set

    Does the clean data set come from the VOC data set? So do you need to search for noise data by yourself? Where is the noise data set used by the author? Is the clean data set put into Celeba_A, and then the noise data set into Celeba_B?

    opened by haolin512900 3
  • The issue in the PascalVoc.py

    The issue in the PascalVoc.py

    As your attempt, you want to divide the pascalvoc dataset into 4, and this .py file is just for pascalvoc dataset. But in your code : def splitlist(folder, trainfolder1, trainfolder2, testfolder): if not os.path.exists(folder): raise Exception('input folder dose not exist ! please check it !!') if not os.path.exists(trainfolder1): os.makedirs(trainfolder1) if not os.path.exists(trainfolder2): os.makedirs(trainfolder2) if not os.path.exists(testfolder): os.makedirs(testfolder)

    filelist = os.listdir(folder)
    sublist = random.sample(filelist, **110000**)
    sublist_a = random.sample(sublist, **55000**)
       .....
    

    The number of pictures in the folder is 110000 and 55000 respectively, but there are only 12031 images in the pascalvoc data set. How did you do it? Or is it that you made a mistake?

    opened by xiaoachen98 1
  • One question in dis_update function

    One question in dis_update function

    in dis_update(self, x_a, x_b, hyperparameters): the content adversial codes are , out_a = self.dis_content(h_a) out_b = self.dis_content(h_b) why not use h_a.detach() and h_b.detach() to avoid the update in the parameters of G ? Thank you!

    opened by laulampaul 0
Code for the Active Speakers in Context Paper (CVPR2020)

Active Speakers in Context This repo contains the official code and models for the "Active Speakers in Context" CVPR 2020 paper. Before Training The c

null 43 Oct 14, 2022
An unofficial implementation of "Unpaired Image Super-Resolution using Pseudo-Supervision." CVPR2020

UnpairedSR An unofficial implementation of "Unpaired Image Super-Resolution using Pseudo-Supervision." CVPR2020 turn RCAN(modified) --> xmodel(xilinx

JiaKui Hu 10 Oct 28, 2022
PyTorch reimplementation of minimal-hand (CVPR2020)

Minimal Hand Pytorch Unofficial PyTorch reimplementation of minimal-hand (CVPR2020). you can also find in youtube or bilibili bare hand youtube or bil

Hao Meng 228 Dec 29, 2022
Code for Referring Image Segmentation via Cross-Modal Progressive Comprehension, CVPR2020.

CMPC-Refseg Code of our CVPR 2020 paper Referring Image Segmentation via Cross-Modal Progressive Comprehension. Shaofei Huang*, Tianrui Hui*, Si Liu,

spyflying 55 Dec 1, 2022
Densely Connected Search Space for More Flexible Neural Architecture Search (CVPR2020)

DenseNAS The code of the CVPR2020 paper Densely Connected Search Space for More Flexible Neural Architecture Search. Neural architecture search (NAS)

Jamin Fong 291 Nov 18, 2022
RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020)

RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020) Hong Wang, Qi Xie, Qian Zhao, and Deyu Meng [PDF] [Supplementary M

Hong Wang 6 Sep 27, 2022
Super Pix Adv - Offical implemention of Robust Superpixel-Guided Attentional Adversarial Attack (CVPR2020)

Super_Pix_Adv Offical implemention of Robust Superpixel-Guided Attentional Adver

DLight 8 Oct 26, 2022
The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory

This repository contains the software implementation of most algorithms used or developed in my research. The LaTeX and Python code for generating the

João Fonseca 3 Jan 3, 2023
Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow 18 Oct 6, 2022
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 29 Dec 28, 2022