When I execute scripts/train.sh yc2 mart, I get this error.

len(train_dataset) gives me 0, yet all the steps prior to the training stage have been respected, and have been successfully completed. How can I solve the problem please? @jayleicn I work on google colab.

Hi,

Thank you for your work. I'm making code to allow for easy testing of caption generation on user videos. Can you tell me how I can generate captions for my own videos?

What features are required? is there a feature generation code I can use? what is the command to run inference on my own videos?

Thanks

First, the train.sh file have some problems that ${dset_name} should be ${model_type} in the Line 39 42 48 50. But when the model_type is 'xl' or 'xlrg', I got the Error:

File "src/train.py", line 639, in main() File "src/train.py", line 635, in main train(model, train_loader, val_loader, device, opt) File "src/train.py", line 330, in train model, training_data, optimizer, ema, device, opt, writer, epoch_i) File "src/train.py", line 78, in train_epoch input_masks_list, token_type_ids_list, input_labels_list) File "src/rtransformer/model.py", line 40, in forward output = self.log_softmax(output[valid_indices])

Training =>: 0%| | 0/313 [00:05<?, ?it/s] Traceback (most recent call last): File "src/train.py", line 639, in main() File "src/train.py", line 635, in main train(model, train_loader, val_loader, device, opt) File "src/train.py", line 330, in train model, training_data, optimizer, ema, device, opt, writer, epoch_i) File "src/train.py", line 131, in train_epoch loss.backward() RuntimeError: Function 'LogSoftmaxBackward' returned nan values in its 0th output.

Pls help me, thanks!

Thank you for your code！ I have a question for you. Can you tell me whether the experimental results of other methods (eg, VTransformer, GVD, AdvInf) in Table 1 and table 2 are obtained from your own experiments or from the corresponding papers ？

Hello, it is me too. I can reduce the results in your paper MART on anet. But the results on yc2 is CIDEr 23.46, not 35.74, Meteor 15.1, not 15.9 and the BLEU@4 is 6.88, not 8. Can you share some parameters different from the anet, which is beneficial for yc2 to a 30+ CIDEr? Thanks very much.

hi thanks for your good work and sharing your code.

when I run your code today, I found that there are some errors during the training. when I run the command "bash scripts/train.sh yc2 mart" it gives me the error like the following:

File "/home/chenj0g/Desktop/mart/recurrent-transformer/src/utils.py", line 18, in load_json with open(file_path, "r") as f: IOError: [Errno 2] No such file or directory: '/home/chenj0g/Desktop/mart/recurrent-transformer/results/yc2_re_init_2021_01_03_15_09_03/model_tmp_greedy_pred_val_lang.json'

do you know how could I solve this problem?

Hi,

Thanks for sharing the code. I found a small error in memory initializer & updater when using MART. The memory has shape [N, L, intermediate_size], but the projection layers are fromhidden_size to hidden_size. If intermediate_size and hidden_size are not equal, it will raise a wrong shape error.

Hello? Thank you for your interesting work. I was trying to generate captioning, however I can't find pretrained model. there Is no pretrained model provided or did I miss something? Do I have to train the whole network only to evaluate the model? Thank you

hi thanks for your good work and sharing your code.

when I run your code today, I found that there are some errors during the training. when I run the command "bash scripts/train.sh yc2 mart" it gives me the error like the following:

File "/home/chenj0g/Desktop/mart/recurrent-transformer/src/utils.py", line 18, in load_json with open(file_path, "r") as f: IOError: [Errno 2] No such file or directory: '/home/chenj0g/Desktop/mart/recurrent-transformer/results/yc2_re_init_2021_01_03_15_09_03/model_tmp_greedy_pred_val_lang.json'

do you know how could I solve this problem?****