Hi, thanks for sharing this great work.

I have a question about the sampling implementation. First of all, what is the difference between diffusion_utils_2 and diffusion_utils_2? I think diffusion_utils_2 is only used for cifar unconditional part. But the difference seems to be larger than that.

Another question is about the difference between current implementation and Algorithm 2 in the paper. If I understand correctly, predict_start_from_noise predicts p(x_0|x_t) and then q_posterior predicts x_{t-1} using equation 7 in the paper. This is different compare to algorithm 2, where equation 11 is used. Are those two equivalent? Or which one is better(stable)?

Thanks!

Hey I have a quick question! Is it possible to do the evaluation loop during training (e.g., every 100 iterations do once of sampling images), instead of having to execute the evaluation separately?

def get_timestep_embedding(timesteps, embedding_dim: int): """ From Fairseq. Build sinusoidal embeddings. This matches the implementation in tensor2tensor, but differs slightly from the description in Section 3.5 of "Attention Is All You Need". """ assert len(timesteps.shape) == 1 # and timesteps.dtype == tf.int32

half_dim = embedding_dim // 2 emb = math.log(10000) / (half_dim - 1)

I don't understand why (half_dim - 1) is used here. According to the transformer's time-coding formula, there should be "emb = math.log(10000) / half_dim", I don't think half_dim should minus 1 here.

Hey :wave:,

maintainers of the DDPM repository. I was wondering if you think it could make sense to advertise a PyTorch version of DDPM on the README here. We are trying to make DDPM as accessible as possible with the diffusers library and are also actively maintaining the modeling code (we're running tests for DDPM on a daily basis). We thought it could make sense to add a link directing to a PyTorch implementation in this README.md - would this make sense to you?

I tried to reproduce DDPM on CIFAR10. As mentioned in the paper, my batchsize is 128, the optimizer is Adam, the learning rate is 0.0002, and I used l2 loss. I found that the training loss kept fluctuating between 0.015 and 0.030. What is this caused by? Should I need to reduce the learning rate? Can you tell me the loss of your training?

Thanks so much for the excellent work and code sharing.

May I inquire about how to compute the rate and distortion in Figure 5 and Table 4 in the paper, especially the rate? Preferably with some codes?

Thanks!

First, thank you for sharing this project with us!

Could you please add an explicit LICENSE file to the repo so that it's clear under what terms the content is provided, and under what terms user contributions are licensed?

Per GitHub docs on licensing:

[...] without a license, the default copyright laws apply, meaning that you retain all rights to your source code and no one may reproduce, distribute, or create derivative works from your work. If you're creating an open source project, we strongly encourage you to include an open source license.

Thanks!