B-cos Networks: Attention is All we Need for Interpretability

Related tags

Deep Learning B-cos
Overview

Convolutional Dynamic Alignment Networks for Interpretable Classifications

M. Böhle, M. Fritz, B. Schiele. B-cos Networks: Alignment is All we Need for Interpretability. CVPR, 2022.

Overview

  • Qualitative Examples

Evaluated on videos

In order to highlight the stability of the contribution-based explanations of the B-cos-Nets, here we provide explanations on videos; for more information, see VideoEvaluation.ipynb.

Comparison to post-hoc methods

Quantitative Interpretability results

In order to reproduce these plots, check out the jupyter notebook Quantitative results. For more information, see the paper and check out the code at interpretability/

Copyright and license

Copyright (c) 2022 Moritz Böhle, Max-Planck-Gesellschaft

This code is licensed under the BSD License 2.0, see license.

Further, you use any of the code in this repository for your research, please cite as:

  @inproceedings{Boehle2022CVPR,
          author    = {Moritz Böhle and Mario Fritz and Bernt Schiele},
          title     = {B-cos Networks: Alignment is All we Need for Interpretability},
          journal   = {IEEE/CVF Conference on Computer Vision and Pattern Recognition ({CVPR})},
          year      = {2022}
      }
Comments
  • Can you help with some of the terms in the paper

    Can you help with some of the terms in the paper

    Hi , thanks for your work. It really open my mind for the interpretable of DNN with a replacement of linenar transform . But when reading the paper ,i have some point not understanding well as below, so if you can help to answear it ? 1.What kind of metric is the 'localisation metric',what 's the physical meaning? (Fig 6) 2.What's the meaning of baseline 'orcale attribution' and 'uniform attribution' ?(Fig 6) image 3.Is there some parameter refer to the area in red ? Or just for better visual ? (Fig 2) image

    opened by scutfrank 2
  • Issues in path and possible memory leak

    Issues in path and possible memory leak

    Hi all, I was trying to run the code but I faced multiple issues:

    1. I am running the experiments/Imagenet/bcos/model.py​, the save_path in the code is still trying to reach the CIFAR10 dataset.

    2. In model.py, get_optimizer():

    "the_model" does not have an attribute called "opti"

    1. after spending some time I managed to solve above issues (not sure if it is gonna harm the functionality), but when I run the code the GPU runs out of memory! I played with the batch_size, but no luck. I guess that is due to a memory leak somewhere. Can anyone verify that?
    opened by pooyanrg 2
  • Where is train.py?

    Where is train.py?

    Hi, Thanks for your interesting paper. I'm trying to train your model but only I can find is Trainer class in training folder and not the training loop code. Can you please add your code for training loop code OR give some guidelines for reproducing your implementation?

    Thanks a lot.

    opened by JuChanSeo 2
  • Transfer Learning

    Transfer Learning

    Hello,

    thanks for your fascinating work. I am trying to use the B-cos network (the densenet121 named “densenet_121_cossched”) in my research but I struggle with having it transfer effectively to smaller datasets, e.g. CUB2011. In fact, it overfits much more ( much worse final test acc) and improves a lot slower than the conventional densenet (In fact, only retraining the final layer leads to no learning whatsoever across a range of hyperparameters that all work for the conventional one). Since you have experience with training this network, I figure I might just ask you:

    • How sensitive was the training to optimized hyperparameters? Do I “just” need to tune regularization etc?
    • Have you done some experiments on transfer learning and maybe figured out effective methods?
    • Could B-cos networks be less suited for fine-grained-classification?
    • How exactly did you train the model? From my point of view your training code does not work, as the trainer class does not get a “loss” argument or am I missing something? I am encountering nan as loss, when using BCE.
    • Did you investigate the impact of norming w in Equation 3 in the paper (maybe with increased B) so that the model rescales the outputs himself?

    Any answers would be greatly appreciated :)

    Greetings

    opened by Gnabe 2
  • Code Running Erro

    Code Running Erro

    By following the README of Training,an error happens when run the code as below. Could you help to check it ,thanks !

    Traceback (most recent call last): File "D:/1.Study/PycharmProjects/Reproduce/B-cos-main/experiments/Imagenet/bcos/model.py", line 93, in start_training(cmd_args, get_model, get_optimizer) File "D:\1.Study\PycharmProjects\Reproduce\B-cos-main\training\training_utils.py", line 17, in start_training training_loop(params, get_model, get_optimizer) File "D:\1.Study\PycharmProjects\Reproduce\B-cos-main\training\training_utils.py", line 83, in training_loop data_handler = Data(params.dataset_name, rank=ddp_rank, File "D:\1.Study\PycharmProjects\Reproduce\B-cos-main\data\data_handler.py", line 82, in init self.train_data_loader = self.init_data_loader(train=True, File "D:\1.Study\PycharmProjects\Reproduce\B-cos-main\data\data_handler.py", line 105, in init_data_loader data = Data.datasets[self.dataset_name](self.data_path, train=train, download=False, File "D:\1.Study\PycharmProjects\Reproduce\B-cos-main\data\datasets.py", line 34, in init path = os.path.join(root, self.base_folder, "train" if train else "val") File "D:\Software\anaconda3\lib\ntpath.py", line 78, in join path = os.fspath(path) TypeError: expected str, bytes or os.PathLike object, not NoneType image

    opened by scutfrank 2
Owner
null
Unofficial PyTorch implementation of Fastformer based on paper "Fastformer: Additive Attention Can Be All You Need"."

Fastformer-PyTorch Unofficial PyTorch implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Usage : import t

Hong-Jia Chen 126 Dec 6, 2022
An implementation of Fastformer: Additive Attention Can Be All You Need in TensorFlow

Fast Transformer This repo implements Fastformer: Additive Attention Can Be All You Need by Wu et al. in TensorFlow. Fast Transformer is a Transformer

Rishit Dagli 139 Dec 28, 2022
Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Fastformer-Keras Unofficial Tensorflow-Keras implementation of Fastformer based on paper Fastformer: Additive Attention Can Be All You Need. Tensorflo

Yam Peleg 10 Jan 30, 2022
A PyTorch implementation of the Transformer model in "Attention is All You Need".

Attention is all you need: A Pytorch Implementation This is a PyTorch implementation of the Transformer model in "Attention is All You Need" (Ashish V

Yu-Hsiang Huang 7.1k Jan 4, 2023
pytorch implementation of Attention is all you need

A Pytorch Implementation of the Transformer: Attention Is All You Need Our implementation is largely based on Tensorflow implementation Requirements N

null 230 Dec 7, 2022
Code and data to accompany the camera-ready version of "Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation" in EMNLP 2021

Code and data to accompany the camera-ready version of "Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation" in EMNLP 2021

Mozhdeh Gheini 16 Jul 16, 2022
Implementation of Vaswani, Ashish, et al. "Attention is all you need."

Attention Is All You Need Paper Implementation This is my from-scratch implementation of the original transformer architecture from the following pape

Brando Koch 195 Dec 30, 2022
[ACM MM 2021] Yes, "Attention is All You Need", for Exemplar based Colorization

Transformer for Image Colorization This is an implemention for Yes, "Attention Is All You Need", for Exemplar based Colorization, and the current soft

Wang Yin 30 Dec 7, 2022
TensorFlow implementation of "Attention is all you need (Transformer)"

[TensorFlow 2] Attention is all you need (Transformer) TensorFlow implementation of "Attention is all you need (Transformer)" Dataset The MNIST datase

YeongHyeon Park 4 Jan 5, 2022
PyTorch implementation of "Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning"

Transparency-by-Design networks (TbD-nets) This repository contains code for replicating the experiments and visualizations from the paper Transparenc

David Mascharka 351 Nov 18, 2022
Official implementation for "Symbolic Learning to Optimize: Towards Interpretability and Scalability"

Symbolic Learning to Optimize This is the official implementation for ICLR-2022 paper "Symbolic Learning to Optimize: Towards Interpretability and Sca

VITA 8 Dec 19, 2022
Adaout is a practical and flexible regularization method with high generalization and interpretability

Adaout Adaout is a practical and flexible regularization method with high generalization and interpretability. Requirements python 3.6 (Anaconda versi

lambett 1 Feb 9, 2022
A human-readable PyTorch implementation of "Self-attention Does Not Need O(n^2) Memory"

memory_efficient_attention.pytorch A human-readable PyTorch implementation of "Self-attention Does Not Need O(n^2) Memory" (Rabe&Staats'21). def effic

Ryuichiro Hataya 7 Dec 26, 2022
Code for "Diffusion is All You Need for Learning on Surfaces"

Source code for "Diffusion is All You Need for Learning on Surfaces", by Nicholas Sharp Souhaib Attaiki Keenan Crane Maks Ovsjanikov NOTE: the linked

Nick Sharp 247 Dec 28, 2022
PixelPick This is an official implementation of the paper "All you need are a few pixels: semantic segmentation with PixelPick."

PixelPick This is an official implementation of the paper "All you need are a few pixels: semantic segmentation with PixelPick." [Project page] [Paper

Gyungin Shin 59 Sep 25, 2022
Per-Pixel Classification is Not All You Need for Semantic Segmentation

MaskFormer: Per-Pixel Classification is Not All You Need for Semantic Segmentation Bowen Cheng, Alexander G. Schwing, Alexander Kirillov [arXiv] [Proj

Facebook Research 1k Jan 8, 2023
Open-Set Recognition: A Good Closed-Set Classifier is All You Need

Open-Set Recognition: A Good Closed-Set Classifier is All You Need Code for our paper: "Open-Set Recognition: A Good Closed-Set Classifier is All You

null 194 Jan 3, 2023
BasicRL: easy and fundamental codes for deep reinforcement learning。It is an improvement on rainbow-is-all-you-need and OpenAI Spinning Up.

BasicRL: easy and fundamental codes for deep reinforcement learning BasicRL is an improvement on rainbow-is-all-you-need and OpenAI Spinning Up. It is

RayYoh 12 Apr 28, 2022
Implementation of ConvMixer-Patches Are All You Need? in TensorFlow and Keras

Patches Are All You Need? - ConvMixer ConvMixer, an extremely simple model that is similar in spirit to the ViT and the even-more-basic MLP-Mixer in t

Sayan Nath 8 Oct 3, 2022