作者您好，打扰了，我在服务器上跑您的代码在evaluation阶段出现了如下报错，请问您遇到过么？期待您的回复！ Traceback (most recent call last): File "train.py", line 353, in scores = evaluate_metrics(model, dict_dataloader_val, text_field) File "train.py", line 62, in evaluate_metrics out, _ = model.beam_search(images, 20, text_field.vocab.stoi[''], 5, out_size=1, File "/mnt/hdd1/alluser/yanjialuo/image-captioning-DLCT/models/captioning_model.py", line 70, in beam_search return bs.apply(visual, out_size, return_probs, **kwargs) File "/mnt/hdd1/alluser/yanjialuo/image-captioning-DLCT/models/beam_search/beam_search.py", line 71, in apply visual, outputs = self.iter(t, visual, outputs, return_probs, **kwargs) File "/mnt/hdd1/alluser/yanjialuo/image-captioning-DLCT/models/beam_search/beam_search.py", line 121, in iter self.model.apply_to_states(self._expand_state(selected_beam, cur_beam_size)) File "/mnt/hdd1/alluser/yanjialuo/image-captioning-DLCT/models/containers.py", line 30, in apply_to_states self._buffers[name] = fn(self._buffers[name]) File "/mnt/hdd1/alluser/yanjialuo/image-captioning-DLCT/models/beam_search/beam_search.py", line 26, in fn s = torch.gather(s.view(*([self.b_s, cur_beam_size] + shape[1:])), 1, RuntimeError: gather_out_cuda(): Expected dtype int64 for index

In my country, the download speed from Baidu is very slow and I can't down needed resources. Can you please upload them to GDrive, OneDrive, or Dropbox? Thank you!

您好，请问您使用https://github.com/facebookresearch/grid-feats-vqa提取的特征维度是固定的吗？我使用[extract_grid_feature.py]提取得到的特征维度不同，所以在batch size训练时，会出现RuntimeError: stack expects each tensor to be equal size, but got [1, 2048, 18, 32] at entry 0 and [1, 2048, 19, 29] at entry 1

/usr/local/lib/python3.7/dist-packages/detectron2/config/config.py in wrapped(self, *args, **kwargs) 188 if _called_with_cfg(*args, **kwargs): 189 explicit_args = _get_args_from_config(from_config_func, *args, **kwargs) --> 190 init_func(self, **explicit_args) 191 else: 192 init_func(self, *args, **kwargs)

TypeError: init() got an unexpected keyword argument 'train_on_pred_boxes' 作者您好，我在复现您论文时，在实现提取特征的代码时出现该问题，我查询了相关的，但是没有解决？

Hello, Thanks for your opening code. When I train this model from the scratch, training loss is down, but testing performances do not change in training stage

Hello, could you explain about the features displayed on net disk? is coco_all_align.hdf5 in the zip file? And what are the files end with z01, z02,z03? I have tried to exact the zip file, but fails.

我尝试使用您的代码自己常见特征集，但是还有部分疑问。region_after和region_before 分别取自proposal_box_features和proposal_box_features1（均值），这有何区别？？ 另外我去https://github.com/facebookresearch/grid-feats-vqa下也并未找到相应区域yaml文件，请问您是否可以分享下您的X-101-region.yaml和X-152-region.yaml文件？？

Traceback (most recent call last): File "/mnt/Pycharm_Remote/DLCT_test/train.py", line 335, in scores = evaluate_metrics(model, dict_dataloader_val, text_field) File "/mnt/Pycharm_Remote/DLCT_test/train.py", line 61, in evaluate_metrics **{'boxes': boxes, 'grids': grids, 'masks': masks}) File "/mnt/Pycharm_Remote/DLCT_test/models/captioning_model.py", line 70, in beam_search return bs.apply(visual, out_size, return_probs, **kwargs) File "/mnt/Pycharm_Remote/DLCT_test/models/beam_search/beam_search.py", line 71, in apply visual, outputs = self.iter(t, visual, outputs, return_probs, **kwargs) File "/mnt/Pycharm_Remote/DLCT_test/models/beam_search/beam_search.py", line 121, in iter self.model.apply_to_states(self._expand_state(selected_beam, cur_beam_size)) File "/mnt/Pycharm_Remote/DLCT_test/models/containers.py", line 30, in apply_to_states self._buffers[name] = fn(self._buffers[name]) File "/mnt/Pycharm_Remote/DLCT_test/models/beam_search/beam_search.py", line 27, in fn beam.expand(*([self.b_s, self.beam_size] + shape[1:]))) RuntimeError: gather_out_cuda(): Expected dtype int64 for index

the beam is float and come from "selected_beam = selected_idx / candidate_logprob.shape[-1]"，so it's float.But index need int.

Hi, in your coding,h5py.File features has keys like ['%d_features' % image_id] , ['%d_grids' % image_id], ['%d_boxes' % image_id], ['%d_size' % image_id], ['%d_mask' % image_id], If I have a new data set, can I just use align.py to get geometric alignment graph after I get grid features and region features using extract_region_feature.py and grid-feats-vqa.

I seems that I have managed to combine the code to generate all the features needed. I want to test it on my custom images without re-training the network. Can you please provide the pretrained checkpoint?

Your script extract_region_feature.py has weights for X-101 hard-coded. But the file with features has name "region_before_X152.hdf5". Also, there is no information from you which checkpoint to use for extracting grid features. In the paper you mention both X-101 and X-152 as extractors.

Which extractor checkpoint should I use for grid-feats-vqa: X-101 or X-152? Can I use X-152 for region features as well?

It is not possible to download the trained pth file from pan.baidu.com It seems we need to install some kind of download software (the Baidu page is in Chinese so it is hard to understand how to proceed).

Could you please host the files in a more user-friendly repository? Like osf.io or Dropbox?

Thx

Thank you so much for sharing your brilliant code with us. But could you share me the code to generate the caption for a image like the Figure 1 in your artical? I would appreciate it if you could share it with me! ^_^