when i feed the input data whose shape is [1, 3, 240, 250], there is an error at https://github.com/ChristophReich1996/Swin-Transformer-V2/blob/d1b89227ef0045c3ab667a4f2cdea9ec4f240236/swin_transformer_v2/model_parts.py#L574 , the error is listed below:

RuntimeError: The size of tensor a (48) must match the size of tensor b (256) at non-singleton dimension 2

it appears that the shape of self.default_grid.repeat_interleave(repeats=offsets.shape[0], dim=0) and offsets is different, I wonder if you have the same problem and it will be of great help if you can help me solve it, thank you~

model.update_resolution(new_window_size=8, new_input_resolution=(512, 640)) ------> RuntimeError: shape '[0, 2, 2, 768, 8, 8]' is invalid for input of size 196608

Hi, I've worked on Swin Transformer v.1 earlier for object detection training and inference. Now I want to improve the result with Swin Transformer v.2. Is it available and is there any way to do that? Many thanks.

源码改建 （当我导入swinv2_tiny_patch4_window8_256.pth，使用窗口为8时候，可以正常跑代码；但是当我导入swinv2_tiny_patch4_window16_256.pth，使用窗口为16时候，导入权重出现不匹配情况；不知道如何处理，请大佬解答一下。 问题如下：） RuntimeError: Error(s) in loading state_dict for Model: size mismatch for model.7.blocks.0.attn.relative_coords_table: copying a param with shape torch.Size([1, 15, 15, 2]) from checkpoint, the shape in current model is torch.Size([1, 31, 31, 2]). size mismatch for model.7.blocks.0.attn.relative_position_index: copying a param with shape torch.Size([64, 64]) from checkpoint, the shape in current model is torch.Size([256, 256]). size mismatch for model.7.blocks.1.attn.relative_coords_table: copying a param with shape torch.Size([1, 15, 15, 2]) from checkpoint, the shape in current model is torch.Size([1, 31, 31, 2]). size mismatch for model.7.blocks.1.attn.relative_position_index: copying a param with shape torch.Size([64, 64]) from checkpoint, the shape in current model is torch.Size([256, 256]).

Hi! I have another question. If I just pip, and then :

class SWIN(nn.Module):
     def __init__(self, num_classes=4):
        super().__init__()
        self.num_classes = num_classes
        # self.pool = nn.MaxPool2d(2, 2)
        self.encoder: SwinTransformerV2 = swin_transformer_v2_t(in_channels=3,
                                                            window_size=8,
                                                            input_resolution=(1024, 1280),
                                                            sequential_self_attention=False,
                                                            use_checkpoint=True)
        self.p=self.encoder.patch_embedding
        self.encoder0 = self.encoder.stages[0]
        ... ...

How to use the checkpoint now? And Is there a pre-trained model for v2_base? (And when I just run like above, a wired problem arises: 'warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")')

window_attention.relative_coordinates_log: copying a param with shape torch.Size([256, 2]) from checkpoint, the shape in current model is torch.Size([4096, 2]).

Hello, I have encountered some small problems when using SwinV2 these days. I would like to get your answers here.

1. when my input size is small, such as 96*96, window_szie=8, will appear https://github.com/ChristophReich1996/Swin-Transformer-V2/blob/cff3824f5d75dfd93553867efcf53562c24dd555/swin_transformer_v2/model_parts.py#L289 RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

2. In this paper, sequential self-attention calculation is used to save GPU memory, but in large-size image application, I set Sequential_self_attention =True, which will result in OOM. Set Sequential_self_attention =False does not.

3. when i update the code https://github.com/ChristophReich1996/Swin-Transformer-V2/blob/cff3824f5d75dfd93553867efcf53562c24dd555/swin_transformer_v2/model_parts.py#L207,GPU memory usage also soared further.

I am currently experimenting with the application of swinV2, an efficient and memory saving network, to 3D data, therefore pay more attention to the occupation problem on the display memory.

The scaled cosine attention part of the implementation seems wrong in model_parts.py

attention_map: torch.Tensor = torch.einsum("bhqd, bhkd -> bhqk", query, key) \
                                      / torch.maximum(torch.norm(query, dim=-1, keepdim=True)
                                                      * torch.norm(key, dim=-1, keepdim=True),
                                                      torch.tensor(1e-06, device=query.device, dtype=query.dtype))

should be corrected as

attention_map: torch.Tensor = torch.einsum("bhqd, bhkd -> bhqk", query, key) \
                                      / torch.maximum(torch.norm(query, dim=-1, keepdim=True)
                                                      @ torch.norm(key, dim=-1, keepdim=True).transpose(-2, -1),
                                                      torch.tensor(1e-06, device=query.device, dtype=query.dtype))

since the equation normalizes the attention values for each query and key pair. The original code would produce a norm vector of shape (B, H, N, 1), while the actual norm matrix we need should be in shape (B, H, N, N).

Thank you very much for your code, it is concise and easy to read, and I would like to ask you a few more questions:

1. Since the output of swin-v2 is a feature map, if I want to use it for semantic segmentation, can I directly upsample to get the final result?
2. If I want to change the dataset, what do I need to do?