Denoising Diffusion Implicit Models

Last update: Jan 5, 2023

Related tags

Overview

Denoising Diffusion Implicit Models (DDIM)

Jiaming Song, Chenlin Meng and Stefano Ermon, Stanford

Implements sampling from an implicit model that is trained with the same procedure as Denoising Diffusion Probabilistic Model, but costs much less time and compute if you want to sample from it (click image below for a video demo):

Running the Experiments

The code has been tested on PyTorch 1.6.

Train a model

Training is exactly the same as DDPM with the following:

python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --doc {MODEL_NAME} --ni

Sampling from the model

Sampling from the generalized model for FID evaluation

python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --doc {MODEL_NAME} --sample --fid --timesteps {STEPS} --eta {ETA} --ni

where

ETA controls the scale of the variance (0 is DDIM, and 1 is one type of DDPM).
STEPS controls how many timesteps used in the process.
MODEL_NAME finds the pre-trained checkpoint according to its inferred path.

If you want to use the DDPM pretrained model:

python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --use_pretrained --sample --fid --timesteps {STEPS} --eta {ETA} --ni

the --use_pretrained option will automatically load the model according to the dataset.

We provide a CelebA 64x64 model here, and use the DDPM version for CIFAR10 and LSUN.

If you want to use the version with the larger variance in DDPM: use the --sample_type ddpm_noisy option.

Sampling from the model for image inpainting

Use --interpolation option instead of --fid.

Sampling from the sequence of images that lead to the sample

Use --sequence option instead.

The above two cases contain some hard-coded lines specific to producing the image, so modify them according to your needs.

References and Acknowledgements

@article{song2020denoising,
  title={Denoising Diffusion Implicit Models},
  author={Song, Jiaming and Meng, Chenlin and Ermon, Stefano},
  journal={arXiv:2010.02502},
  year={2020},
  month={October},
  abbr={Preprint},
  url={https://arxiv.org/abs/2010.02502}
}

This implementation is based on / inspired by:

https://github.com/hojonathanho/diffusion (the DDPM TensorFlow repo),
https://github.com/pesser/pytorch_diffusion (PyTorch helper that loads the DDPM model), and
https://github.com/ermongroup/ncsnv2 (code structure).

Comments

reconstruction from latent code

Hi, thanks for your excellent paper and project !

I want to explore the "5.4 reconstruction from latent code". Can I just reverse the seq and seq_next to produce t and t+1 in the function below to produce the latent code from the input image ?

https://github.com/ermongroup/ddim/blob/51cb290f83049e5381b09a4cc0389f16a4a02cc9/functions/denoising.py#L10

Appreciate any help !

opened by winnechan 6
got wrong FID using pretrained model

Hi, I sampled 50000 images using the provided pretrained CelebA model, but got a FID of 5.83 in the setting of timesteps=1000, eta=0, which is a margin away from 3.51 reported in the paper (when timesteps is set to 100, the FID is 10.13 while 6.53 reported in the paper). May I know where the problem is? Is the total CelebA dataset used for calculating FID?

opened by ztyang196 3
FID of DDPMs on CIFAR-10

Hi,

I found with the converted pretrained CIFAR-10 DDPM,

https://github.com/ermongroup/ddim/blob/34d640e5180cc5ab378f84af6ed596cb0c810e6c/runners/diffusion.py#L220

I got a FID of 5.68 in the setting of timesteps=100, eta=0, which is a margin away from 4.16 reported in the paper. May I know is that about a model you trained by yourselves?

For FID calculation, I use https://github.com/mseitzer/pytorch-fid

opened by GloryyrolG 3
Question in the paper

I have a question in Table1 in this paper.

In the table1, the cases of eta=1 and hat{sigma} are the case of DDPM. However, the performances of these cases are different.

What are the differences between those cases? )

Thank you :

opened by sunwoo76 1
Question in results

Hi, I used your default parameters to train the DDIM model using 1000 diffusion steps, 800k iterations.

But my reproduced results are very bad, can you provide some suggestions?

fidelity --gpu 9 --fid --input1 exp/image_samples/baseline_cifar10/ --input2 cifar10-train --sample_type generalized

FID: 187.58

opened by zzw-zwzhang 1
How to train a better model?

Hi, I use this code to train a new model, but I find that I cannot get similar FID results. You successfully provide a new pretrain model on CelebA, so can you give me some advice? Thanks a lot!

opened by ghost 1
Why not clamp in generalized_steps?

Hi,

Thanks for your great work. May I ask, why you do not clamp in generalized_steps as in ddpm_steps?

https://github.com/ermongroup/ddim/blob/34d640e5180cc5ab378f84af6ed596cb0c810e6c/functions/denoising.py#L24

opened by GloryyrolG 1
Add link to Diffusers library

Hey :wave: from the diffusers team,

Just wanted to ask if you are interested in adding a link to the diffusers library to your README. We're actively maintaining the DDIM pipeline and also enabled use cases to use DDIM with stable diffusion which might be interesting for readers of this repo.

opened by patrickvonplaten 0
sampling code of DDPM

Hi, I found that your provided ddpm code seems to be different from the paper, such as the coefficient of x_t:

(atm1.sqrt() * beta_t) * (1.0 / at).sqrt() * x + ( (1 - beta_t).sqrt() * (1 - atm1)) * x ) / (1.0 - at)

Thanks!

opened by zzw-zwzhang 0
How about the training setting of CelebA model

I am very interested in the training setting of the CelebA model since I retrained one but got a fid of only 4.5 with 1000 steps DDIM sampler. Could you please give me some suggestion about it?

opened by jsttlgdkycy 1
got an unexpected keyword argument `eta`
Thanks for sharing the code!

While I get the following error when I run the demo code of the diffusers.

diffusers/src/diffusers/schedulers/scheduling_ddpm.py line 258 got an unexpected keyword argument `eta`
opened by wyf0912 0
FID of DDPM on CIFAR-10

Hi! :) I wonder with the given configurations, whether we could recover the FID of 3.17 if we use the code to train a DDPM from scratch on CIFAR-10, since I run for 800k steps now and I could only obtain the FID of about 4.

opened by chenweixin107 2
Transferability to transformers

First of all, thank you for providing the code! In the according paper I read that the only difference between DDPM and DDIM is how samples are generated. Intuitively, I would then assume that the CNN model could also be replaced by a transformer-type architecture. If my understanding of your paper is correct, one could still use the same simple loss and your DDIM sample generation. I would highly appreciate your opinion on this. Thanks in advance, Anthony.

opened by anthony-mendil 0
HOW to use the pretrained model of celeba

Special thanks for your outstanding work, I now want to use celeba's pre-trained model, but I don't know how to write the execution command, can you give me a concrete example?

python main.py --config {DATASET}.yml --exp {PROJECT_PATH} --doc {MODEL_NAME} --use_pretrained --sample --fid --timesteps {STEPS} --eta {ETA} -- ni @willieneis @yang-song @jiamings @Zymrael @chenlin9

opened by pokameng 0

Owner

GitHub

Denoising Diffusion Probabilistic Models

Denoising Diffusion Probabilistic Models This repo contains code for DDPM training. Based on Denoising Diffusion Probabilistic Models, Improved Denois

7 Dec 15, 2022

ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (ICCV 2021 Oral)

ILVR + ADM This is the implementation of ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models (ICCV 2021 Oral). This repository is h

225 Dec 28, 2022

Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in Pytorch

Retrieval-Augmented Denoising Diffusion Probabilistic Models (wip) Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in P

55 Jan 1, 2023

BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis

Bilateral Denoising Diffusion Models (BDDMs) This is the official PyTorch implementation of the following paper: BDDM: BILATERAL DENOISING DIFFUSION M

172 Dec 23, 2022

A denoising diffusion probabilistic model (DDPM) tailored for conditional generation of protein distograms

Denoising Diffusion Probabilistic Model for Proteins Implementation of Denoising Diffusion Probabilistic Model in Pytorch. It is a new approach to gen

108 Nov 23, 2022

PyTorch Implementation of DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs

DiffGAN-TTS - PyTorch Implementation PyTorch implementation of DiffGAN-TTS: High

157 Jan 1, 2023

Learning Energy-Based Models by Diffusion Recovery Likelihood

Learning Energy-Based Models by Diffusion Recovery Likelihood Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, Diederik P. Kingma Paper: https://arxiv.o

41 Nov 22, 2022

This is the codebase for Diffusion Models Beat GANS on Image Synthesis.

3k Dec 26, 2022

Official PyTorch implementation for FastDPM, a fast sampling algorithm for diffusion probabilistic models

Official PyTorch implementation for "On Fast Sampling of Diffusion Probabilistic Models". FastDPM generation on CIFAR-10, CelebA, and LSUN datasets. S

68 Dec 26, 2022

Codebase for Diffusion Models Beat GANS on Image Synthesis.

128 Dec 2, 2022

McGill Physics Hackathon 2021: Reaction-Diffusion Models for the Generation of Biological Patterns

DiffuseAnimals: Reaction-Diffusion Models for the Generation of Biological Patterns Introduction Reaction-diffusion equations can be utilized in order

2 Mar 7, 2022

High-Resolution Image Synthesis with Latent Diffusion Models

Latent Diffusion Models Requirements A suitable conda environment named ldm can be created and activated with: conda env create -f environment.yaml co

5.6k Jan 4, 2023

Official codebase for running the small, filtered-data GLIDE model from GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models.

GLIDE This is the official codebase for running the small, filtered-data GLIDE model from GLIDE: Towards Photorealistic Image Generation and Editing w

2.9k Jan 4, 2023