Hi there, A brilliant work! Thanks.

I would be more grateful if you can provide the inference.py, sometimes also called predict.py, by which I can generate image of any input sentence.

Could you please tell me how to put the image data into a folder when I want to use FID, LPIPS, or IS? The train set and data set are split by ".pikle" files and I don't know how to the evaluation metric.

Hello, author. At the beginning of the training, the system asked me for the netG model of the 600th training. After I downloaded the model you gave me, the program continued to run, but stopped after only a few minutes. Ask why you need the model for the 600th run to start training and why the program stops after a few minutes. Thanks to the author. 我的英文不太好，加一个我原本的意思。 作者你好，在开始训练时系统向我索要第600次训练的netG模型。我在您所给的模型下载下来后，程序可以继续运行，但只运行了几分钟就会停下了。请问为什么开始训练需要第600次的模型，还有为什么程序运行几分钟就会停下来。感谢作者。

@wtliao

I wanted to calculate R-precision for comparisons with few state-of-the-art benchmarks so I adjusted the code into main.py to calculate R-precision using already given finetuned trained model by you. I achieved 73% value around. I was just checking your paper newest (5th) version where you have reported R precision 86%. Can you please share your code to calculate the R-precision so that I can figure out why we have large differences in our R precision scores.

Thank you

@wtliao

I have calculated the scores:

• FID score but I am getting very high value : FID: 73.33472569962976
• IS score is quite low : Inception mean: 4.732609 , Inception std: 0.1345223

For now, I am generating the images from epoch 550

Is the best FID score is calculated from last epoch 550 ? or Should I calculate the IS and FID score of the checkpoints every 10 or 50 epochs (or let me know what epoch I should use) and then choose the checkpoints with best FID and IS score ?

Traceback (most recent call last): File "main.py", line 410, in image_encoder = CNN_ENCODER(cfg.TEXT.EMBEDDING_DIM) File "/data01/hxf/text2image/DAMSM.py", line 127, in init model = models.inception_v3(pretrained=True, transform_input=False) File "/home/omnisky/hxf/lib/python3.6/site-packages/torchvision/models/inception.py", line 53, in inception_v3 progress=progress) File "/home/omnisky/hxf/lib/python3.6/site-packages/torch/hub.py", line 557, in load_state_dict_from_url return torch.load(cached_file, map_location=map_location) File "/home/omnisky/hxf/lib/python3.6/site-packages/torch/serialization.py", line 608, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/home/omnisky/hxf/lib/python3.6/site-packages/torch/serialization.py", line 777, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '<'.

@wtliao I want to train the CUB and COCO models by myself instead of using provided trained model (onedrive repo). Would model.py help me to train the model ? or is there something more I should follow ?

Hi, thank you for your excellent releasing code! I have a little question, how can I visualize the predicted mask maps when validation. I attempted to use the save_image() API from torchvision.utils, but the results are not consistent with the provided mask maps from the paper.

Looking forward to your reply! Thank you!

Hello, I am a freshman, I want to run this program. But when things got tough.(FileNotFoundError: [Errno 2] No such file or directory: '../data/birds/captions.pickle'). How should this step be solved, thank you.

Traceback (most recent call last): File "D:\text2image-main\text2image-main\main.py", line 496, in dataset = TextDataset(cfg.DATA_DIR, 'test', File "D:\text2image-main\text2image-main\datasets.py", line 124, in init self.wordtoix, self.n_words = self.load_text_data(data_dir, split) File "D:\text2image-main\text2image-main\datasets.py", line 243, in load_text_data x = pickle.load(f) EOFError: Ran out of input

i find something ways to solve , while set "num_workers = 0". but this way cannot solve. please help

Hi,

Thanks for providing the code for the paper!

I tried to reproduce the FID score on the COCO dataset by generating images for the validation dataset as reported in the paper using the generator for 120th epoch (netG_120.pth) and text encoder for 595th epoch. I used the Pytorch implementation of the FID score and it gives me around 121 FID score as opposed to 19 reported.

I had resized the original COCO images to 256x256 resolution to have a consistent image size but the score is still high.

Even the generation is weird for sample sentences.

For eg.

The image attached has caption "A close up of a boat on a field with a cloudy sky". This caption has been taken from the paper but the generation using the final generator model and text encoder model is nowhere near presented in the paper.

Any suggestions from your side as to what has to be done?

Also can you please mention the difference between main.py and main_finetune.py? There is not much of a difference in both these scripts.

Hello，I test the trained models from your onedrive in COCO，the FID score is 28.1929. It has a big gap with the paper data (19.37) . My FID code is from https://github.com/bioinf-jku/TTUR and my GPU is 4 3090 . I'm confused about this and want to know the parameters that your FID code sets about COCO, such as the image size of groundtruth，whether or not you split the coco data and so on.Thanks