Denoising Diffusion Implicit Models

Related tags

Deep Learning ddim
Overview

Denoising Diffusion Implicit Models (DDIM)

Jiaming Song, Chenlin Meng and Stefano Ermon, Stanford

Implements sampling from an implicit model that is trained with the same procedure as Denoising Diffusion Probabilistic Model, but costs much less time and compute if you want to sample from it (click image below for a video demo):

Running the Experiments

The code has been tested on PyTorch 1.6.

Train a model

Training is exactly the same as DDPM with the following:

python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --doc {MODEL_NAME} --ni

Sampling from the model

Sampling from the generalized model for FID evaluation

python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --doc {MODEL_NAME} --sample --fid --timesteps {STEPS} --eta {ETA} --ni

where

  • ETA controls the scale of the variance (0 is DDIM, and 1 is one type of DDPM).
  • STEPS controls how many timesteps used in the process.
  • MODEL_NAME finds the pre-trained checkpoint according to its inferred path.

If you want to use the DDPM pretrained model:

python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --use_pretrained --sample --fid --timesteps {STEPS} --eta {ETA} --ni

the --use_pretrained option will automatically load the model according to the dataset.

We provide a CelebA 64x64 model here, and use the DDPM version for CIFAR10 and LSUN.

If you want to use the version with the larger variance in DDPM: use the --sample_type ddpm_noisy option.

Sampling from the model for image inpainting

Use --interpolation option instead of --fid.

Sampling from the sequence of images that lead to the sample

Use --sequence option instead.

The above two cases contain some hard-coded lines specific to producing the image, so modify them according to your needs.

References and Acknowledgements

@article{song2020denoising,
  title={Denoising Diffusion Implicit Models},
  author={Song, Jiaming and Meng, Chenlin and Ermon, Stefano},
  journal={arXiv:2010.02502},
  year={2020},
  month={October},
  abbr={Preprint},
  url={https://arxiv.org/abs/2010.02502}
}

This implementation is based on / inspired by:

Comments
  • reconstruction from latent code

    reconstruction from latent code

    Hi, thanks for your excellent paper and project !

    I want to explore the "5.4 reconstruction from latent code". Can I just reverse the seq and seq_next to produce t and t+1 in the function below to produce the latent code from the input image ?

    https://github.com/ermongroup/ddim/blob/51cb290f83049e5381b09a4cc0389f16a4a02cc9/functions/denoising.py#L10

    Appreciate any help !

    opened by winnechan 6
  • got wrong FID using pretrained model

    got wrong FID using pretrained model

    Hi, I sampled 50000 images using the provided pretrained CelebA model, but got a FID of 5.83 in the setting of timesteps=1000, eta=0, which is a margin away from 3.51 reported in the paper (when timesteps is set to 100, the FID is 10.13 while 6.53 reported in the paper). May I know where the problem is? Is the total CelebA dataset used for calculating FID?

    opened by ztyang196 3
  • FID of DDPMs on CIFAR-10

    FID of DDPMs on CIFAR-10

    Hi,

    I found with the converted pretrained CIFAR-10 DDPM,

    https://github.com/ermongroup/ddim/blob/34d640e5180cc5ab378f84af6ed596cb0c810e6c/runners/diffusion.py#L220

    I got a FID of 5.68 in the setting of timesteps=100, eta=0, which is a margin away from 4.16 reported in the paper. May I know is that about a model you trained by yourselves?

    For FID calculation, I use https://github.com/mseitzer/pytorch-fid

    opened by GloryyrolG 3
  • Question in the paper

    Question in the paper

    I have a question in Table1 in this paper.

    In the table1, the cases of eta=1 and hat{sigma} are the case of DDPM. However, the performances of these cases are different.

    What are the differences between those cases? 화면 캡처 2022-06-15 161148 )

    Thank you :

    opened by sunwoo76 1
  • Question in results

    Question in results

    Hi, I used your default parameters to train the DDIM model using 1000 diffusion steps, 800k iterations.

    But my reproduced results are very bad, can you provide some suggestions?

    fidelity --gpu 9 --fid --input1 exp/image_samples/baseline_cifar10/ --input2 cifar10-train --sample_type generalized

    FID: 187.58

    opened by zzw-zwzhang 1
  • How to train a better model?

    How to train a better model?

    Hi, I use this code to train a new model, but I find that I cannot get similar FID results. You successfully provide a new pretrain model on CelebA, so can you give me some advice? Thanks a lot!

    opened by ghost 1
  • Why not clamp in generalized_steps?

    Why not clamp in generalized_steps?

    Hi,

    Thanks for your great work. May I ask, why you do not clamp in generalized_steps as in ddpm_steps?

    https://github.com/ermongroup/ddim/blob/34d640e5180cc5ab378f84af6ed596cb0c810e6c/functions/denoising.py#L24

    opened by GloryyrolG 1
  • Add link to Diffusers library

    Add link to Diffusers library

    Hey :wave: from the diffusers team,

    Just wanted to ask if you are interested in adding a link to the diffusers library to your README. We're actively maintaining the DDIM pipeline and also enabled use cases to use DDIM with stable diffusion which might be interesting for readers of this repo.

    opened by patrickvonplaten 0
  • sampling code of DDPM

    sampling code of DDPM

    Hi, I found that your provided ddpm code seems to be different from the paper, such as the coefficient of x_t:

    (atm1.sqrt() * beta_t) * (1.0 / at).sqrt() * x + ( (1 - beta_t).sqrt() * (1 - atm1)) * x ) / (1.0 - at)

    image

    Thanks!

    opened by zzw-zwzhang 0
  • How about the training setting of CelebA model

    How about the training setting of CelebA model

    I am very interested in the training setting of the CelebA model since I retrained one but got a fid of only 4.5 with 1000 steps DDIM sampler. Could you please give me some suggestion about it?

    opened by jsttlgdkycy 1
  • got an unexpected keyword argument `eta`

    got an unexpected keyword argument `eta`

    Thanks for sharing the code!

    While I get the following error when I run the demo code of the diffusers.

    diffusers/src/diffusers/schedulers/scheduling_ddpm.py line 258 got an unexpected keyword argument `eta`
    opened by wyf0912 0
  • FID of DDPM on CIFAR-10

    FID of DDPM on CIFAR-10

    Hi! :) I wonder with the given configurations, whether we could recover the FID of 3.17 if we use the code to train a DDPM from scratch on CIFAR-10, since I run for 800k steps now and I could only obtain the FID of about 4.

    opened by chenweixin107 2
  • Transferability to transformers

    Transferability to transformers

    First of all, thank you for providing the code! In the according paper I read that the only difference between DDPM and DDIM is how samples are generated. Intuitively, I would then assume that the CNN model could also be replaced by a transformer-type architecture. If my understanding of your paper is correct, one could still use the same simple loss and your DDIM sample generation. I would highly appreciate your opinion on this. Thanks in advance, Anthony.

    opened by anthony-mendil 0
  • HOW to use the pretrained model of celeba

    HOW to use the pretrained model of celeba

    Special thanks for your outstanding work, I now want to use celeba's pre-trained model, but I don't know how to write the execution command, can you give me a concrete example?

    python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --doc {MODEL_NAME} --use_pretrained --sample --fid --timesteps {STEPS} --eta {ETA} -- ni @willieneis @yang-song @jiamings @Zymrael @chenlin9

    opened by pokameng 0
Owner
null
GitHub
Denoising Diffusion Probabilistic Models

Denoising Diffusion Probabilistic Models This repo contains code for DDPM training. Based on Denoising Diffusion Probabilistic Models, Improved Denois

Alexander Markov 7 Dec 15, 2022
ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (ICCV 2021 Oral)

ILVR + ADM This is the implementation of ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (ICCV 2021 Oral). This repository is h

Jooyoung Choi 225 Dec 28, 2022
Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in Pytorch

Retrieval-Augmented Denoising Diffusion Probabilistic Models (wip) Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in P

Phil Wang 55 Jan 1, 2023
BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis

Bilateral Denoising Diffusion Models (BDDMs) This is the official PyTorch implementation of the following paper: BDDM: BILATERAL DENOISING DIFFUSION M

null 172 Dec 23, 2022
A denoising diffusion probabilistic model (DDPM) tailored for conditional generation of protein distograms

Denoising Diffusion Probabilistic Model for Proteins Implementation of Denoising Diffusion Probabilistic Model in Pytorch. It is a new approach to gen

Phil Wang 108 Nov 23, 2022
PyTorch Implementation of DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs

DiffGAN-TTS - PyTorch Implementation PyTorch implementation of DiffGAN-TTS: High

Keon Lee 157 Jan 1, 2023
Learning Energy-Based Models by Diffusion Recovery Likelihood

Learning Energy-Based Models by Diffusion Recovery Likelihood Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, Diederik P. Kingma Paper: https://arxiv.o

Ruiqi Gao 41 Nov 22, 2022
This is the codebase for Diffusion Models Beat GANS on Image Synthesis.

This is the codebase for Diffusion Models Beat GANS on Image Synthesis.

OpenAI 3k Dec 26, 2022
Official PyTorch implementation for FastDPM, a fast sampling algorithm for diffusion probabilistic models

Official PyTorch implementation for "On Fast Sampling of Diffusion Probabilistic Models". FastDPM generation on CIFAR-10, CelebA, and LSUN datasets. S

Zhifeng Kong 68 Dec 26, 2022
Codebase for Diffusion Models Beat GANS on Image Synthesis.

Codebase for Diffusion Models Beat GANS on Image Synthesis.

Katherine Crowson 128 Dec 2, 2022
McGill Physics Hackathon 2021: Reaction-Diffusion Models for the Generation of Biological Patterns

DiffuseAnimals: Reaction-Diffusion Models for the Generation of Biological Patterns Introduction Reaction-diffusion equations can be utilized in order

Austin Szuminsky 2 Mar 7, 2022
High-Resolution Image Synthesis with Latent Diffusion Models

Latent Diffusion Models Requirements A suitable conda environment named ldm can be created and activated with: conda env create -f environment.yaml co

CompVis Heidelberg 5.6k Jan 4, 2023
Official codebase for running the small, filtered-data GLIDE model from GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models.

GLIDE This is the official codebase for running the small, filtered-data GLIDE model from GLIDE: Towards Photorealistic Image Generation and Editing w

OpenAI 2.9k Jan 4, 2023
Code for the paper "Implicit Representations of Meaning in Neural Language Models"

Implicit Representations of Meaning in Neural Language Models Preliminaries Create and set up a conda environment as follows: conda create -n state-pr

Belinda Li 39 Nov 3, 2022
Pseudo-Visual Speech Denoising

Pseudo-Visual Speech Denoising This code is for our paper titled: Visual Speech Enhancement Without A Real Visual Stream published at WACV 2021. Autho

Sindhu 94 Oct 22, 2022
EDCNN: Edge enhancement-based Densely Connected Network with Compound Loss for Low-Dose CT Denoising

EDCNN: Edge enhancement-based Densely Connected Network with Compound Loss for Low-Dose CT Denoising By Tengfei Liang, Yi Jin, Yidong Li, Tao Wang. Th

workingcoder 115 Jan 5, 2023
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021)

Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021, official Pytorch implementatio

Microsoft 247 Dec 25, 2022
HiFi-GAN: High Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

HiFiGAN Denoiser This is a Unofficial Pytorch implementation of the paper HiFi-GAN: High Fidelity Denoising and Dereverberation Based on Speech Deep F

Rishikesh (ऋषिकेश) 134 Dec 27, 2022
Official implementation of Deep Convolutional Dictionary Learning for Image Denoising.

DCDicL for Image Denoising Hongyi Zheng*, Hongwei Yong*, Lei Zhang, "Deep Convolutional Dictionary Learning for Image Denoising," in CVPR 2021. (* Equ

Z80 91 Dec 21, 2022