MAE-pytorch/modeling_pretrain.py", line 296, in pretrain_mae_base_patch16_224
    **kwargs)
TypeError: __init__() got an unexpected keyword argument 'num_classes'

分享一个issue，我一开始用的是11.22更新的requirements.txt，会报如下错误：

File"/MAE-pytorch/modeling_pretrain.py", line 319, in pretrain_mae_base_patch16_224 **kwargs) **kwargs) TypeError: init() got an unexpected keyword argument 'in_chans'

我发现好像是timm==0.3.2导致的，后来升至0.4.12可以解决这个问题。 测试了V100 (CentOS) 和 A100 (Ubuntu)，都存在这个issue。 卸载timm好像也一样可以跑……我第一次看CV的code，了解的还不是很多。 顺便想问一下你们在V100选用的batch size是64吗

Hello, when I was carrying out the visualization, the loading model failed. Could you please tell me the reason? I fine-tune the model with pre-training, visualizing calls for fine-tuning the model

Hello, thanks for your implementation.

I have read the main part of your code, but I didn't find the code that controls the Partial fine-tuning. Could you please tell me where is that part in "run_class_finetuning.py", "modeling_finetune.py" or anywhere else?

Wainting for your reply, thank you.

Hi,

I'm very impressed with the quick reproduction, nice work!

I have tried running inference with the provided models and noticed that the current checkpoints do not contain the positional embedding. This is not an issue when running on the same resolution (224,224). However, it makes it tricky to run inference at higher resolutions, since there is no positional encoding to interpolate.

I have been using the following hard-coded workaround but as far as I can see, the only solution is to change the model so that the positional encoding is stored as part of the checkpoint. Here is the hard-coded solution for loading current models and running tuning/inference:

# this replaces the code from line 334 in run_class_finetuning.py

        # Maybe interpolate position embedding
        old_n_positions = int((224/16)**2)
        if model.pos_embed.shape[1] != old_n_positions:
            embedding_size = model.pos_embed.shape[-1]
            old_pos_embed = modeling_finetune.get_sinusoid_encoding_table(old_n_positions, embedding_size)
            num_patches = model.patch_embed.num_patches
            num_extra_tokens = model.pos_embed.shape[-2] - num_patches
            assert num_extra_tokens == 0, "No support for class tokens"
            # height (== width) for the checkpoint position embedding
            orig_size = int((old_pos_embed.shape[-2] - num_extra_tokens) ** 0.5)
            # height (== width) for the new position embedding
            new_size = int(num_patches ** 0.5)
            # class_token and dist_token are kept unchanged
            if orig_size != new_size:
                print("Position interpolate from %dx%d to %dx%d" % (orig_size, orig_size, new_size, new_size))
                extra_tokens = old_pos_embed[:, :num_extra_tokens]
                # only the position tokens are interpolated
                pos_tokens = old_pos_embed[:, num_extra_tokens:]
                pos_tokens = pos_tokens.reshape(-1, orig_size, orig_size, embedding_size).permute(0, 3, 1, 2)
                pos_tokens = torch.nn.functional.interpolate(
                    pos_tokens, size=(new_size, new_size), mode='bicubic', align_corners=False)
                pos_tokens = pos_tokens.permute(0, 2, 3, 1).flatten(1, 2)
                new_pos_embed = torch.cat((extra_tokens, pos_tokens), dim=1)
                model.pos_embed = new_pos_embed

I want to know how to process the dataset.... There are a lot of tar files.

Is there any commands or processing python file? It is my firtst time to do this....

Sorry for my stupid question..

On two gpus.

Epoch: [24] [1230/1251] eta: 0:00:06 lr: 0.000375 min_lr: 0.000375 loss: 0.6870 (0.6848) loss_scale: 2097152.0000 (2046895.3111) weight_decay: 0.0500 (0.0500) grad_norm: 0.0929 (0.0969) time: 0.3023 data: 0.0010 max mem: 8361 Epoch: [24] [1240/1251] eta: 0:00:03 lr: 0.000375 min_lr: 0.000375 loss: 0.6877 (0.6848) loss_scale: 2097152.0000 (2047300.2804) weight_decay: 0.0500 (0.0500) grad_norm: 0.0942 (0.0971) time: 0.2731 data: 0.0018 max mem: 8361 Epoch: [24] [1250/1251] eta: 0:00:00 lr: 0.000375 min_lr: 0.000375 loss: 0.6856 (0.6849) loss_scale: 2097152.0000 (2047698.7754) weight_decay: 0.0500 (0.0500) grad_norm: 0.0942 (0.0971) time: 0.2560 data: 0.0012 max mem: 8361 Epoch: [24] Total time: 0:06:23 (0.3067 s / it) Averaged stats: lr: 0.000375 min_lr: 0.000375 loss: 0.6856 (0.6851) loss_scale: 2097152.0000 (2047698.7754) weight_decay: 0.0500 (0.0500) grad_norm: 0.0942 (0.0971) Epoch: [25] [ 0/1251] eta: 1:25:25 lr: 0.000375 min_lr: 0.000375 loss: 0.6770 (0.6770) loss_scale: 2097152.0000 (2097152.0000) weight_decay: 0.0500 (0.0500) grad_norm: 0.0918 (0.0918) time: 4.0974 data: 3.7792 max mem: 8361 Epoch: [25] [ 10/1251] eta: 0:13:50 lr: 0.000375 min_lr: 0.000375 loss: 0.6854 (0.6838) loss_scale: 2097152.0000 (2097152.0000) weight_decay: 0.0500 (0.0500) grad_norm: 0.0910 (0.0949) time: 0.6694 data: 0.3704 max mem: 8361

How does the phenomenon occur?

Hello, I'm a novice. I want to ask. Download directly ImageNet_ILSVRC2012 dataset, can you get the pre training model according to your pre train process? If not, how can I get it. Can two GPUs pre train themselves?

I am beginner and crazy about this MAE, but about the pretrained model--pretrain_mae_vit_base_mask_0.75_400e.pth downloaded from BaiDu yun, I really wanna know where to load it in the file. Can it be used in modeling_pretrain.py?

Thanks for your code! It's really a great work! I want to train MAE on a dataset with 512*512 image size, and how can I adjust the model structure? Thanks for your help.

Traceback (most recent call last): File "run_mae_vis.py", line 138, in main(opts) File "run_mae_vis.py", line 79, in main model = get_model(args) File "run_mae_vis.py", line 63, in get_model model = create_model( File "/home/ub/miniconda3/envs/torch1.8/lib/python3.6/site-packages/timm/models/factory.py", line 57, in create_model model = create_fn(**model_args, **kwargs) File "/home/ub/bwj/MAE-pytorch/modeling_pretrain.py", line 317, in pretrain_mae_base_patch16_224 **kwargs) TypeError: init() got an unexpected keyword argument 'num_classes'

I'm running mae's finetune"return torch.layer_norm(input, normalized_shape, weight, bias, eps, torch.backends.cudnn.enabled) RuntimeError: Given normalized_shape=[768], expected input with shape [*, 768], but got input of size[12]",Please tell me what the problem is.

Hey,

Thanks very much for this excellent repo, it is definitely worth hundreds of thousands of stars! :)

I wonder which datasets are used for the released pretrained model? Since it works too well to be true for image recovery from randomly masked inputs from FFHQ datasets, especially the backgrounds which has no clues at all for the input part. So is it the case that the model actually saw the data somewhere?

Thank you very much in advance!!

Dear Author,

Firstly thanks and appreciated for your great contribution.

While fine tuning with IN1K base on the pre-train model which also trained with IN1K, the result is similar to the paper's as follows:

But if I fine tune with iNaturalist with the same pre-train model and same finetune parameters listed in your github page, the result is really bad as follows:

So, what do you think of the possible reason for me? Looking forward to your reply, Thanks in advance !

BTW, the picture amount of iNaturalist is about 260,000, icludes 1010 classes. The train data and val data is not separated in iNaturalist, I divided follow the ratio of IN1K(96% for train, 4% for val).

In addition, do you have the plan to implement fine tuning code of object detection and semantic segmentation ? If yes, how much longer we need wait for ? Thanks again !