It seems that the code cannot generate "./train_pkl/samples_bytes_0.pkl" successfully.

The first running, generate pkl, but return error:

./train_pkl/samples_bytes_0.pkl global_rank 0 cached 0/1281167 takes 0.00s per block global_rank 0 cached 128116/1281167 takes 21.24s per block global_rank 0 cached 256232/1281167 takes 19.01s per block global_rank 0 cached 384348/1281167 takes 18.36s per block global_rank 0 cached 512464/1281167 takes 29.66s per block global_rank 0 cached 640580/1281167 takes 35.94s per block global_rank 0 cached 768696/1281167 takes 36.32s per block global_rank 0 cached 896812/1281167 takes 35.54s per block global_rank 0 cached 1024928/1281167 takes 37.19s per block global_rank 0 cached 1153044/1281167 takes 46.55s per block global_rank 0 cached 1281160/1281167 takes 50.39s per block Traceback (most recent call last): File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 260, in main() File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 256, in main cmd=cmd) subprocess.CalledProcessError: Command '['/home/ma-user/anaconda3/envs/Pytorch-1.4.0/bin/python', '-u', 'main.py', '--local_rank=0', '--cfg', 'configs/as_base_patch4_shift5_224.yaml', '--data-path', '/cache/imagenet/imagenet/', '--eval', '--resume', '/cache/model/asmlp_base_patch4_shift5_224.pth', '--moxfile', '0']' died with <Signals.SIGKILL: 9>.

The second running, pkl contain nothing, so the error occurs:

./train_pkl/samples_bytes_0.pkl Traceback (most recent call last): File "main.py", line 349, in main(config) File "main.py", line 78, in main dataset_train, dataset_val, data_loader_train, data_loader_val, mixup_fn = build_loader(config) File "/home/ma-user/work/AS-MLP-main/data/build.py", line 17, in build_loader dataset_train, config.MODEL.NUM_CLASSES = build_dataset(is_train=True, config=config) File "/home/ma-user/work/AS-MLP-main/data/build.py", line 80, in build_dataset cache_mode=config.DATA.CACHE_MODE if is_train else 'part') File "/home/ma-user/work/AS-MLP-main/data/cached_image_folder.py", line 250, in init cache_mode=cache_mode) File "/home/ma-user/work/AS-MLP-main/data/cached_image_folder.py", line 122, in init self.init_cache() File "/home/ma-user/work/AS-MLP-main/data/cached_image_folder.py", line 137, in init_cache self.samples = pickle.load(handle) EOFError: Ran out of input Traceback (most recent call last): File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 260, in main() File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/torch/distributed/launch.py", line 256, in main cmd=cmd) subprocess.CalledProcessError: Command '['/home/ma-user/anaconda3/envs/Pytorch-1.4.0/bin/python', '-u', 'main.py', '--local_rank=0', '--cfg', 'configs/as_base_patch4_shift5_224.yaml', '--data-path', '/cache/imagenet/imagenet/', '--eval', '--resume', '/cache/model/asmlp_base_patch4_shift5_224.pth', '--moxfile', '0']' returned non-zero exit status 1.

Like many projects, they would provide torch cuda extensions and a setup.py to compile and generate .so file so that they can be loaded and used. Could you provide such method to use as_shift op?

Hello, thanks for sharing your work. We have tride the code on AS-MLP-T, and only get 81.13/81.06 in two seperate experiments. Are there any settings needed to be modified?

When using train_asmlp_b.sh, the above error occurs:

File "/home/ma-user/work/AS-MLP-main/models/as_mlp.py", line 165, in forward x = shortcut + self.drop_path(x) File "/home/ma-user/anaconda3/envs/Pytorch-1.4.0/lib/python3.6/site-packages/apex/amp/wrap.py", line 58, in wrapper return orig_fn(*new_args, **kwargs) RuntimeError: The size of tensor a (56) must match the size of tensor b (60) at non-singleton dimension 3

Thanks for sharing such a simple yet effective backbone work~

When I run the code, I find no noticeable runtime difference between using the shift_cuda.py and using Algorithm 1 in the paper with Pytorch itself.

Do I miss something to activate the CUDA acceleration?

In fact, I copy part of the core codes in this repo and integrate the codes to the Swin Transformer, instead of running this repo directly.

Could you please help me with this problem? Thanks~

Hello! Thank you for your wonderful job! it is said that as-mlp uses the same training strategy in Swin Transformer, I want to know to what extend the work use the same strategy ? For example, does as-mlp use Mixup、Cutmix?
I saw there exists codes for mixup in function build_loader which requires config.AUG.MIXUP, but I didn't find config.AUG in the config file for models. I want to know the detailed training setting for your models, may you do the favor for me?

The CUDA kernel of basic op seems to be borrowed from https://github.com/d-li14/involution, especially implied by https://github.com/svip-lab/AS-MLP/blob/main/models/shift_cuda.py#L169.

Please consider properly giving credit to the involution repo and citing us. Thanks.

Hi, I have a few questions about the dimensions in yourt code.

• Is your input dimension format like B C H W, eg, 1, 3, 224, 224?

• If so, I suppose the dim in layernorm you initialized is C. when I run this, I got an error like Given normalized_shape=[2304], expected input with shape [*, 2304], but got input of size[8, 2304, 8, 8]

• And also, how do you get the shape of nW*B, window_size*window_size, C after axis shift?

Thank you

https://github.com/svip-lab/AS-MLP/blob/946848d957e5b1c33226ddacaa35f422a103ddfb/models/as_mlp.py#L187-L222

Thanks for sharing your work!

Why did you design PathMerging Layer normalizing features first?

I think that if we normalize features after reduce layer, we can reduce the size of model and flops.

Did you do ablation study for this layer?

For example,

 class PatchMerging(nn.Module): 
     r""" Patch Merging Layer. 
  
     Args: 
         input_resolution (tuple[int]): Resolution of input feature. 
         dim (int): Number of input channels. 
         norm_layer (nn.Module, optional): Normalization layer.  Default: nn.LayerNorm 
     """ 
  
     def __init__(self, input_resolution, dim, norm_layer=nn.LayerNorm): 
         super().__init__() 
         self.input_resolution = input_resolution 
         self.dim = dim 
         self.reduction = nn.Conv2d(4 * dim, 2 * dim, 1, 1, bias=False) 
         self.norm = norm_layer(2 * dim) 
  
     def forward(self, x): 
         """ 
         x: B, H*W, C 
         """ 
         B, C, H, W = x.shape 
         #assert L == H * W, "input feature has wrong size" 
         assert H % 2 == 0 and W % 2 == 0, f"x size ({H}*{W}) are not even." 
  
         x = x.view(B, C, H, W) 
  
         x0 = x[:, :, 0::2, 0::2]  # B C H/2 W/2  
         x1 = x[:, :, 1::2, 0::2]  # B C H/2 W/2  
         x2 = x[:, :, 0::2, 1::2]  # B C H/2 W/2  
         x3 = x[:, :, 1::2, 1::2]  # B C H/2 W/2  
         x = torch.cat([x0, x1, x2, x3], 1)  # B 4*C H/2 W/2  

         x = self.reduction(x) 
         x = self.norm(x) 
         return x 

Thank you for your work! However, I did not find your code for semantic segmentation. I am currently working on pure-MLP segmentation and it would be a huge help if you could provide the ASMLP code for semantic segmentation.

I arrived this repo via followings' likes. For those who like me, it would be good if paper link is included in README. Anyway, thanks for your great effort and reports.

Sincerely, Sungguk Cha