Thank you very much for your work. I tried to use your method to conduct pre-training on my data set, and then compared with the initialization model and imagenet supervised training model respectively. The results showed that the convergence speed of SimMIM pre-training model was similar to the initialization model, although the accuracy would be gradually better than the initialization model after several iterations. But not as good as the convergence of the supervised training model. Is your method as described by MAE in Fine-tune : still improves accuracy after many iterations? Looking forward to your reply.

Hi! This is a wonderful work. After reading the paper, my intuition is that the input image will be masked for reconstruction. Then, while I read the code, I was surprised to find that you did not mask the input image directly, rather you masked the features maps output by Patch_Embed. See here https://github.com/microsoft/SimMIM/blob/519ae7b0999b9d720daa61e3848cd41b8fbd9978/models/simmim.py#L36

I was wondering what is the reasoning behind this approach and how is it different from the more intuitive one of masking the original image directly. Thank you.

As far as I know, for ViT experiment: you use random mask with patch size 32. But when running fine-tuning, you use a default patch size of 16. I got confused on how you tackle the mismatch in <patch_size> between these. As I dig into your utils.py, the load_pretrained function only resolves the mismatch patch size for Relative Position Embeddings. Please correct me if I miss some things?

Hi, Thanks for the amazing work and share the oss code! I was wondering if you could share the reason of this?

MODEL.VIT.USE_RPB: True in simmim_finetune__vit_base__img224__800ep.yaml MODEL.VIT.USE_SHARED_RPB: True in simmim_pretrain__vit_base__img224__800ep.yaml

Do we expect use the same RPB between pretrain and finetune?

Thanks,

thanks for sharing your excellent work！ i have a question about data augment in pretraining： have you try other data augment like RandomResizedCrop() + RandomHorizontalFlip() + RandomVerticalFlip() or other compose？ and RandomResizedCrop() + RandomHorizontalFlip() work the best?

Hi thanks for sharing your work. I was wondering if someone can provide we with some additional information about the following line (and the two lines after that):

https://github.com/microsoft/SimMIM/blob/519ae7b0999b9d720daa61e3848cd41b8fbd9978/models/vision_transformer.py#L78

Why exactly do we subtract 3, 2 and 1 respectively? And isn't the statement on line 80 overwriting the statement two lines prior on line 78?

What exactly does the relative_position_index matrix hold?

For Swin Transformer, the default masked patch size is 32x32, but the first patch embedding layer needs 4x4 patch size. So I want know if the learned masked token is designed for 4x4 patch size, but you also use 32x32 masking strategy?

Hi~, any plan to release the codes of DDP training of swinV2-G with multi machine in SimMIM？ we want to do some research on big model(swinV2-G with 3B parameters) based on your research. Thanks for replay~

Hi @caoyue10 @ancientmooner Thanks a lot for your wonderful job! And I have a question about the AvgDist metric in the paper. In the experiment, you mention the "AvgDist" which is a newly proposed metric. However, I haven't found the reference of the metric in the paper or implemention in the code. So could you please give me any suggestion about this metric?

Again, thanks a lot for your contributions~

Hi, thanks for the great codes! Just wondering if you have any plan to support the 3D version of SimMIM (Video Swin Masked Encoding)? I think there is some paper about video MAE but it is not in Swin styles...

Nice work! I want to know the specific configs or code for semantic segmentation in ADE20K and object detection in COCO. When would you release the configs or code for these experiments?