The repository contains reproducible PyTorch source code of our paper Generative Modeling with Optimal Transport Maps, ICLR 2022.

Overview

Generative Modeling with Optimal Transport Maps

The repository contains reproducible PyTorch source code of our paper Generative Modeling with Optimal Transport Maps, ICLR 2022. It focuses on Optimal Transport Modeling (OTM) in ambient space, e.g. spaces of high-dimensional images. While analogous approaches consider OT maps in the latent space of an autoencoder, this paper focuses on fitting an OT map directly between noise and ambient space. The method is evaluated on generative modeling and unpaired image restoration tasks. In particular, large-scale computer vision problems, such as denoising, colorization, and inpainting are considered in unpaired image restoration. The overall pipeline of OT as generative map and OT as cost of generative model is given below.

Latent Space Optimal Transport

Our method is different from the prevalent approach of OT in the latent space shown below.

Ambient Space Mass Transport

The scheme of our approach for learning OT maps between unequal dimensions.

Prerequisites

The implementation is GPU-based. Single GPU (V100) is enough to run each experiment. Tested with torch==1.4.0 torchvision==0.5.0. To reproduce the reported results, consider using the exact version of PyTorch and its required dependencies as other versions might be incompatible.

Repository structure

All the experiments are issued in the form of pretty self-explanatory python codes.

Main Experiments

Execute the following commands in the source folder.

Training

  • python otm_mnist_32x22.py --train 1 -- OTM between noise and MNIST, 32x32, Grayscale;
  • python otm_cifar_32x32.py --train 1 -- OTM between noise and CIFAR10, 32x32, RGB;
  • python otm_celeba_64x64.py --train 1 -- OTM between noise and CelebA, 64x64, RGB;
  • python otm_celeba_denoise_64x64.py --train 1 -- OTM for unpaired denoising on CelebA, 64x64, RGB;
  • python otm_celeba_colorization_64x64.py --train 1 -- OTM for unpaired colorization on CelebA, 64x64, RGB;
  • python otm_celeba_inpaint_64x64.py --train 1 -- OTM unpaired inpainting on CelebA, 64x64, RGB.

Run inference with the best iteration.

Inference

  • python otm_mnist_32x32.py --inference 1 --init_iter 100000
  • python otm_cifar_32x32.py --inference 1 --init_iter 100000
  • python otm_celeba_64x64.py --inference 1 --init_iter 100000
  • python otm_celeba_denoise_64x64.py --inference 1 --init_iter 100000
  • python otm_celeba_colorization_64x64.py --inference 1 --init_iter 100000
  • python otm_celeba_inpaint_64x64.py --inference 1 --init_iter 100000

Toy Experiments in 2D

  • source/toy/OTM-GO MoG.ipynb -- Mixture of 8 Gaussians;
  • source/toy/OTM-GO Moons.ipynb -- Two Moons;
  • source/toy/OTM-GO Concentric Circles.ipynb -- Concentric Circles;
  • source/toy/OTM-GO S Curve.ipynb -- S Curve;
  • source/toy/OTM-GO Swirl.ipynb -- Swirl.

Refer to Credit Section for baselines.

Results

Optimal transport modeling between high-dimensional noise and ambient space.

Randomly generated samples

Optimal transport modeling for unpaired image restoration tasks.

Following is the experimental setup that is considered for unpaired image restoration.

OTM for image denoising on test C part of CelebA, 64 × 64.

OTM for image colorization on test C part of CelebA, 64 × 64.

OTM for image inpainting on test C part of CelebA, 64 × 64.

Optimal transport modeling for toy examples.

OTM in low-dimensional space, 2D.

Credits

You might also like...
This repository contains the code for the CVPR 2021 paper
This repository contains the code for the CVPR 2021 paper "GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields"

GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields Project Page | Paper | Supplementary | Video | Slides | Blog | Talk If

Code for "Retrieving Black-box Optimal Images from External Databases" (WSDM 2022)

Retrieving Black-box Optimal Images from External Databases (WSDM 2022) We propose how a user retreives an optimal image from external databases of we

A PyTorch implementation of ICLR 2022 Oral paper PiCO: Contrastive Label Disambiguation for Partial Label Learning
A PyTorch implementation of ICLR 2022 Oral paper PiCO: Contrastive Label Disambiguation for Partial Label Learning

PiCO: Contrastive Label Disambiguation for Partial Label Learning This is a PyTorch implementation of ICLR 2022 Oral paper PiCO; also see our Project

Official codebase for ICLR oral paper Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling

CLIORA This is the official codebase for ICLR oral paper: Unsupervised Vision-Language Grammar Induction with Shared Structure Modeling. We introduce

Approaches to modeling terrain and maps in python
Approaches to modeling terrain and maps in python

topography 🌎 Contains different approaches to modeling terrain and topographic-style maps in python Features Inverse Distance Weighting (IDW) A given

This repository contains a pytorch implementation of
This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametric Head Model (CVPR 2022)".

HeadNeRF: A Real-time NeRF-based Parametric Head Model This repository contains a pytorch implementation of "HeadNeRF: A Real-time NeRF-based Parametr

Minimal PyTorch implementation of Generative Latent Optimization from the paper
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Imposter-detector-2022 - HackED 2022 Team 3IQ - 2022 Imposter Detector
Imposter-detector-2022 - HackED 2022 Team 3IQ - 2022 Imposter Detector

HackED 2022 Team 3IQ - 2022 Imposter Detector By Aneeljyot Alagh, Curtis Kan, Jo

This repo contains the pytorch implementation for Dynamic Concept Learner (accepted by ICLR 2021).
This repo contains the pytorch implementation for Dynamic Concept Learner (accepted by ICLR 2021).

DCL-PyTorch Pytorch implementation for the Dynamic Concept Learner (DCL). More details can be found at the project page. Framework Grounding Physical

Comments
  • Plots issue

    Plots issue

    Hi, Thanks for the repo, I think you forgot to include src.plotters There is no implementation for: from src.plotters import plot_noise_interp_unequal, plot_inv_noise_interp_unequal

    opened by amoazeni75 1
Owner
Litu Rout
I am broadly interested in Optimization, Statistical Learning Theory, Interactive Machine Learning, and Optimal Transport.
Litu Rout
Code for paper "Vocabulary Learning via Optimal Transport for Neural Machine Translation"

**Codebase and data are uploaded in progress. ** VOLT(-py) is a vocabulary learning codebase that allows researchers and developers to automaticaly ge

null 416 Nov 25, 2022
Transport Mode detection - can detect the mode of transport with the help of features such as acceeration,jerk etc

title emoji colorFrom colorTo sdk app_file pinned Transport_Mode_Detector ?? purple yellow gradio app.py false Configuration title: string Display tit

Nishant Rajadhyaksha 3 Jan 16, 2022
POT : Python Optimal Transport

POT: Python Optimal Transport This open source Python library provide several solvers for optimization problems related to Optimal Transport for signa

Python Optimal Transport 1.7k Nov 28, 2022
Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted)

NLOS-OT Official implementation of NLOS-OT: Passive Non-Line-of-Sight Imaging Using Optimal Transport (IEEE TIP, accepted) Description In this reposit

Ruixu Geng(耿瑞旭) 15 Nov 28, 2022
Universal Probability Distributions with Optimal Transport and Convex Optimization

Sylvester normalizing flows for variational inference Pytorch implementation of Sylvester normalizing flows, based on our paper: Sylvester normalizing

Rianne van den Berg 171 Dec 1, 2022
PyTorch implementation for Score-Based Generative Modeling through Stochastic Differential Equations (ICLR 2021, Oral)

Score-Based Generative Modeling through Stochastic Differential Equations This repo contains a PyTorch implementation for the paper Score-Based Genera

Yang Song 709 Dec 2, 2022
Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology Sharon Zhou, Eric Zelikman

Stanford Machine Learning Group 34 Nov 16, 2022
[CVPR 2022] CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation

CoTTA Code for our CVPR 2022 paper Continual Test-Time Domain Adaptation Prerequisite Please create and activate the following conda envrionment. To r

Qin Wang 83 Nov 25, 2022
Non-Homogeneous Poisson Process Intensity Modeling and Estimation using Measure Transport

Non-Homogeneous Poisson Process Intensity Modeling and Estimation using Measure Transport This GitHub page provides code for reproducing the results i

Andrew Zammit Mangion 1 Nov 8, 2021
This repository contains the source code of our work on designing efficient CNNs for computer vision

Efficient networks for Computer Vision This repo contains source code of our work on designing efficient networks for different computer vision tasks:

Sachin Mehta 386 Nov 26, 2022