Hi there,

Thanks for your excellent work! I am trying to reproduce the results of VQ-Diffusion-S on CUB-200 with the provided configs. But the trained model cannot generate high-fidelity images and results in a FID score of more than 30.

I checked the code and dataset but do not locate the problem. Can you give me some suggestions to reproduce the results, like which hyper-parameters should I try to change?

Thanks a lot!

Hi there, I am interested in your excellent work and try to understand your code. Can you provide the config file to run inference on Figure 5 (Text guided image editing by VQ-Diffusion.)? It will be very helpful. Thanks!

Thank you for releasing the code of this excellent work!

Regarding the below function, I couldn't figure out the usage of the q_pred function in L215 and L237.

https://github.com/cientgu/VQ-Diffusion/blob/37bbcccdd4aef1794dac645128d864a9f69ed985/image_synthesis/modeling/transformers/diffusion_transformer.py#L206

I understand q_pred as a function that takes an initial state and a time, and returns a distribution with noise at that time. However, the q_pred function in L215 receives log_x_t instead of log_x_start, while the comment says it returns q(xt|x0). In addition, I would be grateful if you would tell me which equation q_pred in L237 corresponds to.

https://github.com/cientgu/VQ-Diffusion/blob/37bbcccdd4aef1794dac645128d864a9f69ed985/image_synthesis/modeling/transformers/diffusion_transformer.py?_pjax=%23js-repo-pjax-container#L186 https://github.com/cientgu/VQ-Diffusion/blob/37bbcccdd4aef1794dac645128d864a9f69ed985/image_synthesis/modeling/transformers/diffusion_transformer.py?_pjax=%23js-repo-pjax-container#L240

As shown in line186, you predict x_0 from x_t at any timestep with transformer model. In the line240 for inference, x_t -> x_{t-1}, you predict x_0 with p(x_0|x_t), and then predict x_{t-1} using q_posterior q(x_{t-1}|x_t, x_0) function.

So why don't directly predict the x_0 with p(x_0|x_T) in the inference?

Hi, thanks for the excellent implementation!

Could you please tell me what is the purpose of the 'filter_ratio' parameter in the sampling function? And I also note that the intermediate training results are sampled with different filter_ratio parameters. How should we interpret the results with different values? Thanks!

Hi, can you share how to use multiple GPUs to train the code? According to you scripts in running_command folder, it seems only support use one single GPU.

Hi , thanks for your code. I want to run your code on CUB200 dataset. I download this dataset from http://www.vision.caltech.edu/visipedia/CUB-200.html. But I find that it does not incude filenames.pickle files. Furthermore the annotation of this dataset is .mat format, but in your dataset code, the anotation seems .txt format. (this_text_path = os.path.join(data_root, 'text', 'text', name+'.txt')). So I want to ask whether you can tell me how to get the CUB200 dataset as your description.

Looking forward to your reply.

Thanks for releasing the codes of this awesome work! May I know the training cost of the VQ-Diffusion-B model? How long does the training take when using 8 V100 GPUs?

Nice work, and thanks to your kindful share! But I wonder the GPU requirement for the training and how to switch the VQ-Diffusion-S for the training? Looking forward to your reply!

Hi author, thanks for sharing your inspiring work!

I found that some codes could be hard to follow. For example, in line 182, what is the goal of this log_add_exp function, and how this calculation of log_probs corresponds to equations in the paper? It seems that you implemented VQ diffusion all in log scale, which is different from the original DDPM to a large extent. Could you explain Why?

Thanks. Looking forward to your reply.

Hi, I want to ask whether the code supports to use multi-machine and multi-gpu. In fact, I want to use two machines, each machine includes 8 GPUs. But when I try to use them with your code, I find only 8 GPU is used.

Hi, In your code, you use the following code to set whether using null text vector: is_empty_text = torch.logical_not(input['condition_mask'][:, 2]).unsqueeze(1).unsqueeze(2).repeat(1, 77, 512) But, I found that if all of the caption length larger than 2, is_empty_text will always be False. So, I want to ask how to control the classifier free guidance? Whether we add some <image, null text> pair to the training dataset?

Hi Authors,

Thanks for sharing this nice work! I am trying to reproduce the results of unconditional synthesis on FFHQ dataset. Compared to other tasks, however, training and inference details for this experiment seem to be insufficient. Could you tell me the training details and inference code for unconditional image generation on FFHQ?

Thanks a lot:)

Hi. The current version of the code seems to work with 256X256 input/output images. I am wondering if there is any way to modify the size of input and output images.

Thanks,

Hi, could you please release the pretrained model on Oxford flower dataset? Meanwhile, did you follow the previous work which split the flower dataset into 82 classes in training data and 20 classes (class 1 to 20) in testing data? Thanks very much!