[ICCV 2021] Released code for Causal Attention for Unbiased Visual Recognition

Related tags

Deep Learning CaaM
Overview

CaaM

This repo contains the codes of training our CaaM on NICO/ImageNet9 dataset. Due to my recent limited bandwidth, this codebase is still messy, which will be further refined and checked recently.

0. Bibtex

If you find our codes helpful, please cite our paper:

@inproceedings{wang2021causal,
  title={Causal Attention for Unbiased Visual Recognition},
  author={Wang, Tan and Zhou, Chang and Sun, Qianru and Zhang, Hanwang},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2021}
}

1. Preparation

  1. Installation: Python3.6, Pytorch1.6, tensorboard, timm(0.3.4), scikit-learn, opencv-python, matplotlib, yaml
  2. Dataset:
  1. Please remember to change the data path in the config file.

2. Evaluation:

  1. For ResNet18 on NICO dataset
CUDA_VISIBLE_DEVICES=0 python train.py -cfg conf/ours_resnet18_multilayer2_bf0.02_noenv_pw5e5.yaml -debug -gpu -eval pretrain_model/nico_resnet18_ours_caam-best.pth

The results will be: Val Score: 0.4638461470603943 Test Score: 0.4661538600921631

  1. For T2T-ViT7 on NICO dataset
CUDA_VISIBLE_DEVICES=0,1 python train.py -cfg conf/ours_t2tvit7_bf0.02_s4_noenv_pw5e4.yaml -debug -gpu -multigpu -eval pretrain_model/nico_t2tvit7_ours_caam-best.pth

The results will be: Val Score: 0.3799999952316284 Test Score: 0.3761538565158844

  1. For ImageNet-9 dataset

Similarly, the pretrained model is in pretrain_model. Please note that on ImageNet9, we report the best performance for the 3 metrics in our paper. The pretrained model is for bias and unbias and we did not save the model for the best ImageNet-A.

3. Train

To perform training, please run the sh file in scripts. For example:

sh scripts/run_baseline_resnet18.sh

4. An interesting finding

Recently I found an interesting thing by accident. The mixup added on the baseline model would not bring much performance improvements (see Table 1. in the main paper). However, when performing mixup based on our CaaM, the performance can be further boosted.

Specifically, you can active the mixup by:

sh scripts/run_ours_resnet18_mixup.sh

This can make our CaaM achieve about 50~51% Val & Test accuracy on NICO dataset.

Acknowledgement

Special thanks to the authors of ReBias and IRM, and the datasets used in this research project.

If you have any question or find any bug, please kindly email me.

You might also like...
Allele-specific pipeline for unbiased read mapping(WIP), QTL discovery(WIP), and allelic-imbalance analysis

WASP2 (Currently in pre-development): Allele-specific pipeline for unbiased read mapping(WIP), QTL discovery(WIP), and allelic-imbalance analysis Requ

Pytorch implementation of the AAAI 2022 paper "Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification"

[AAAI22] Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification We point out the overlooked unbiasedness in long-tailed clas

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones
Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

HaloNet - Pytorch Implementation of the Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones. This re

NP DRAW paper released code
NP DRAW paper released code

NP-DRAW: A Non-Parametric Structured Latent Variable Model for Image Generation This repo contains the official implementation for the NP-DRAW paper.

Code of PVTv2 is released! PVTv2 largely improves PVTv1 and works better than Swin Transformer with ImageNet-1K pre-training.
Code of PVTv2 is released! PVTv2 largely improves PVTv1 and works better than Swin Transformer with ImageNet-1K pre-training.

Updates (2020/06/21) Code of PVTv2 is released! PVTv2 largely improves PVTv1 and works better than Swin Transformer with ImageNet-1K pre-training. Pyr

Released code for Objects are Different: Flexible Monocular 3D Object Detection, CVPR21

MonoFlex Released code for Objects are Different: Flexible Monocular 3D Object Detection, CVPR21. Work in progress. Installation This repo is tested w

This repository contains code released by Google Research.

This repository contains code released by Google Research.

Code for "Causal autoregressive flows" - AISTATS, 2021

Code for "Causal Autoregressive Flow" This repository contains code to run and reproduce experiments presented in Causal Autoregressive Flows, present

Code for NeurIPS 2021 paper: Invariant Causal Imitation Learning for Generalizable Policies

Invariant Causal Imitation Learning for Generalizable Policies Ioana Bica, Daniel Jarrett, Mihaela van der Schaar Neural Information Processing System

Comments
  • Question for the partition weights learning

    Question for the partition weights learning

    Thank you sharing this amazing work.

    I have one question for the partition weight \theta learning in Eq(7), we may claim that the parameter \theta is learning mappings from image ids to the context(environment) space in \mathbb{R}^M.

    I am just curious that if it is possible to directly learn a function that maps from the original feature space \mathcal{X} to the context space, such that theta will not be parameter matrix but a parameterized model?

    I assume it could be, however a difference between the current implementation and parameterized model is, we may need multiple model for each X. As the paper has shown that CAAM can be plugged in any intermediate layers in the models, thus for different intermediate X, we need to learn different parameterized models. Different from that, a global parameter matrix \theta is more efficient, since it can be shared across different intermediate representation of X. Please let me know if there is anything missing.

    Thank you in advance.

    opened by rowedenny 0
Owner
Wang Tan
Ph.D. student of MreaL Lab, NTU
Wang Tan
Toward Spatially Unbiased Generative Models (ICCV 2021)

Toward Spatially Unbiased Generative Models Implementation of Toward Spatially Unbiased Generative Models (ICCV 2021) Overview Recent image generation

Jooyoung Choi 88 Dec 1, 2022
Simple Tensorflow implementation of Toward Spatially Unbiased Generative Models (ICCV 2021)

Spatial unbiased GANs — Simple TensorFlow Implementation [Paper] : Toward Spatially Unbiased Generative Models (ICCV 2021) Abstract Recent image gener

Junho Kim 16 Apr 15, 2022
LBK 26 Dec 28, 2022
PyTorch code for ICLR 2021 paper Unbiased Teacher for Semi-Supervised Object Detection

Unbiased Teacher for Semi-Supervised Object Detection This is the PyTorch implementation of our paper: Unbiased Teacher for Semi-Supervised Object Detection

Facebook Research 366 Dec 28, 2022
Official Pytorch implementation of "Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning (NeurIPS 2021)

Unbiased Classification Through Bias-Contrastive and Bias-Balanced Learning (NeurIPS 2021) Official Pytorch implementation of Unbiased Classification

Youngkyu 17 Jan 1, 2023
[ICCV 2021] Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification

Counterfactual Attention Learning Created by Yongming Rao*, Guangyi Chen*, Jiwen Lu, Jie Zhou This repository contains PyTorch implementation for ICCV

Yongming Rao 90 Dec 31, 2022
Pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering".

TRAnsformer Routing Networks (TRAR) This is an official implementation for ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visu

Ren Tianhe 49 Nov 10, 2022
This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

?? ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

Hyungtae Lim 225 Dec 29, 2022
This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.

A three-stage detection and recognition pipeline of complex meters in wild This is the first released system towards detection and recognition of comp

Yan Shu 19 Nov 28, 2022
Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

ArtFlow Official PyTorch implementation of the paper: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows Jie An*, Siyu Huang*, Yibing

null 123 Dec 27, 2022