Hi @jacobgil!

Thank you for this amazing piece of work. I was wondering if you plan to open-source the code to try out your experiments on Google's ViT (An Image is Worth ...) as well. If it's already there inside the repo, could you point me to it?

Update: I was able to use timm and make use of the ViT model it comes with:

timm_vit_model = timm.create_model('vit_large_patch16_384', pretrained=True)
timm_vit_model.eval()
roller = VITAttentionGradRollout(timm_vit_model, discard_ratio=0.9)
mask = roller(x.unsqueeze(0), label_idx)

However, I am still a bit unsure as to how to actually visualize the mask. Could you help?

Should there be a condition to allow users to pass the head_fusion method in grad_rollout()?

Something like -

...
    for attention, grad in zip(attentions, gradients):                
            weights = grad
            if head_fusion == "mean":
                attention_heads_fused = (attention*weights).mean(axis=1)
            elif head_fusion == "max":
                attention_heads_fused = (attention*weights).max(axis=1)[0]
            elif head_fusion == "min":
                 attention_heads_fused = (attention*weights).min(axis=1)[0]
            else:
                raise "Attention head fusion type Not supported"
            
            attention_heads_fused[attention_heads_fused < 0] = 0
...

How do we change the code when the input image's shape is reshaped to, for example, (256, 128). It seems that the code is only satisfied a square image

Hi @jacobgil, I am using this project for my swin transformers but it is giving error showing

     35             # print("a : ",a)
     36             # print(a.size())
---> 37             a = a / a.sum(dim=-1)
     38 
     39             result = torch.matmul(a, result)

RuntimeError: The size of tensor a (49) must match the size of tensor b (64) at non-singleton dimension 1 ```

Please a give a solution for the same

There's a tiny mistake in the function show_mask_on_image. The heatmap you get from cv2 is in bgr format, so you need to convert it to rgb before adding to the img:

def show_mask_on_image(img, mask):
    ...
    heatmap = cv2.cvtColor(heatmap, cv2.COLOR_BGR2RGB)
    cam = heatmap + np.float32(img)
    ...

while applying the attention rollout on a finetuned timm vit model (base_patch32_224) I'm am getting the following error with input tensor of shape: torch.Size([1, 3, 224, 224]):

RuntimeError Traceback (most recent call last) in () ----> 1 mask_1 = attention_rollout(test_image_1_tensor)

8 frames in reshape_transform(tensor, height, width) 1 def reshape_transform(tensor, height=7, width=7): 2 result = tensor[:, 1 : , :].reshape(tensor.size(0), ----> 3 height, width, tensor.size(2)) 4 5 # Bring the channels to the first dimension,

RuntimeError: shape '[1, 7, 7, 7]' is invalid for input of size 37583

Kindly advice on how to properly apply on the model as I'm facing the same issue for FullGrad in [pytorch-grad-cam] on the same model.

Hello @jacobgil ,

Do u have any recommendations on evaluating explainable methods using tools like Quantus . https://arxiv.org/pdf/2202.06861.pdf

Best Regards, @jaiswati

Hi Thanks for sharing nice work.

I noticed that you've done normalizing attention score (row sum to 1) as mentioned in the original attention rollout paper.

I = torch.eye(attention_heads_fused.size(-1))
a = (attention_heads_fused + 1.0*I)/2
a = a / a.sum(dim=-1)

But it seems when dividing the summation of row attention score, keepdim=True should be apply to ensure that sum of row attention score after normalization should be 1.

a = a / a.sum(dim=-1,keepdim=True)

Maybe I'm wrong, please double check this issue. Thanks