[CVPR 2021] Released code for Counterfactual Zero-Shot and Open-Set Visual Recognition

Related tags

Deep Learning gcm-cf
Overview

Counterfactual Zero-Shot and Open-Set Visual Recognition

This project provides implementations for our CVPR 2021 paper Counterfactual Zero-Shot and Open-Set Visual Recognition, where we propose a counterfactual-based binary seen/unseen classifier (GCM-CF) for Zero-Shot Learning (ZSL) and Open-Set Recognition (OSR). This repo contains

  • ZSL: Strong binary seen/unseen classifier that is plug-and-play with any ZSL method
  • ZSL: Integrations with TF-VAEGAN, RelationNet, GDAN, CADA-VAE, LisGAN, AREN
  • OSR: Complete the OSR code base on MNIST, SVHN, CIFAR10, CIFAR+10, CIFAR+50 with 5 fixed random seed
  • OSR: Strong baseline (F1 score) of Softmax, OpenMax, CGDL
  • OSR: Implementation of our GCM-CF

For technical details, please refer to:

Counterfactual Zero-Shot and Open-Set Visual Recognition
Zhongqi Yue*, Tan Wang*, Hanwang Zhang, Qianru Sun, Xian-Sheng Hua
* Equal contribution
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

Usage

Please refer to the README.md in ZSL and OSR folder, respectively.

TODO

Citation

If you find our work or the code useful, please consider cite our paper using:

@inproceedings{yue2021counterfactual,
  title={Counterfactual Zero-Shot and Open-Set Visual Recognition},
  author={Yue, Zhongqi and Wang, Tan and Zhang, Hanwang and Sun, Qianru and Hua, Xian-Sheng},
  booktitle= {CVPR},
  year={2021}
}
You might also like...
This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.
This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.

A three-stage detection and recognition pipeline of complex meters in wild This is the first released system towards detection and recognition of comp

Yan Shu 19 Nov 28, 2022
OpenGAN: Open-Set Recognition via Open Data Generation
OpenGAN: Open-Set Recognition via Open Data Generation

OpenGAN: Open-Set Recognition via Open Data Generation ICCV 2021 (oral) Real-world machine learning systems need to analyze novel testing data that di

Shu Kong 90 Jan 6, 2023
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022
EMNLP 2021 Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections

Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections Ruiqi Zhong, Kristy Lee*, Zheng Zhang*, Dan Klein EMN

Ruiqi Zhong 42 Nov 3, 2022
[CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator
[CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator

involution Official implementation of a neural operator as described in Involution: Inverting the Inherence of Convolution for Visual Recognition (CVP

Duo Li 1.3k Dec 28, 2022
PyTorch reimplementation of the paper Involution: Inverting the Inherence of Convolution for Visual Recognition [CVPR 2021].

Involution: Inverting the Inherence of Convolution for Visual Recognition Unofficial PyTorch reimplementation of the paper Involution: Inverting the I

Christoph Reich 100 Dec 1, 2022
Implementation of
Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

null 105 Nov 7, 2022
[CVPR 21] Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.
[CVPR 21] Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.

Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting, CVPR 2021. Ayan Kumar Bhunia, Pinaki nath Chowdhury, Yongxin Yan

Ayan Kumar Bhunia 44 Dec 12, 2022
Comments
  • Dataset link has expired

    Dataset link has expired

    Hello, in the data set division link you gave（https://www.mpi-inf.mpg.de/departments/computer-vision-and-machine-learning/research/zero-shot-learning/zero-shot-learning-the-good-the-bad-and-the-ugly/ ）, I click the link in the page but can't download it.

    opened by CandiceHan123 1
  • How configure the size of the image?

    How configure the size of the image?

    I want to change the size of the image to 64*64, but I encountered some problems: """ GARBAGE40 Experiment: GARBAGE40-baseline-lamda100 Begin to Run Exp 0... decreasing_lr: [60, 100, 150] Training... Epoch = 0 Traceback (most recent call last): File "D:/a全部文件/科研 竞赛 项目/南京理工研究生项目/新类检测/OSR/基于DNN的OSR方法/Code/复现/gcm-cf-main/gcm-cf-main/osr/lvae_train.py", line 461, in best_val_loss, best_val_epoch = train(args, lvae) File "D:/a全部文件/科研 竞赛 项目/南京理工研究生项目/新类检测/OSR/基于DNN的OSR方法/Code/复现/gcm-cf-main/gcm-cf-main/osr/lvae_train.py", line 130, in train loss, mu, output, output_mu, x_re, rec, kl, ce = lvae.loss(data, target, target_en, next(beta), args.lamda, args) File "D:\a全部文件\科研 竞赛 项目\南京理工研究生项目\新类检测\OSR\基于DNN的OSR方法\Code\复现\gcm-cf-main\gcm-cf-main\osr\model.py", line 342, in loss pmu2_2, pvar2_2, pmu2_1, pvar2_1, pmu1_2, pvar1_2, pmu1_1, pvar1_1 = self.lnet(x, y_de, args) File "D:\a全部文件\科研 竞赛 项目\南京理工研究生项目\新类检测\OSR\基于DNN的OSR方法\Code\复现\gcm-cf-main\gcm-cf-main\osr\model.py", line 243, in lnet predict = F.log_softmax(self.classifier(latent_y), dim=1) File "C:\Users\296714435\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "C:\Users\296714435\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\modules\linear.py", line 96, in forward return F.linear(input, self.weight, self.bias) File "C:\Users\296714435\AppData\Local\Programs\Python\Python37\lib\site-packages\torch\nn\functional.py", line 1847, in linear return torch._C._nn.linear(input, weight, bias) RuntimeError: CUDA error: CUBLAS_STATUS_INVALID_VALUE when calling cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc) ** On entry to SGEMM parameter number 10 had an illegal value """

    opened by Nicky282 2
  • TF-VAEGAN Results

    TF-VAEGAN Results

    Dear Authors,

    Thank you for the wonderful work. I was trying to replicate the results of the TF-VAEGAN model mentioned in your paper using this code. I am training the TF-VAEGAN model from scratch on the PS2.0 dataset. I am not able to replicate the results mentioned in your paper. Can you suggest to me, how I can replicate the results using the train_tfvaegan_inductive.py file?

    Thank you for your valuable time.

    opened by animesh-007 0
  • Counterfactual Images

    Counterfactual Images

    I've managed to recreate the results on SVHN and other dataset. I got 0.7905 f1 score on SVHN, which is close to the values on the paper. But when I saved the counterfactual images, they are all gray and I can not recreate the images on the paper. I just run the SVHN_cf.script without any extra arguments. Is there anything hyperparameters or arguments I should use to get the same counterfactual images on the paper? Thank you very much! Screenshot from 2021-04-19 17-22-37

    opened by zxl101 1
Owner
null
GitHub
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR2022)[paper] Authors: Chenhang He, Ruihuang Li, Shuai Li, L

Billy HE 141 Dec 30, 2022
[ICCV 2021] Counterfactual Attention Learning for Fine-Grained Visual Categorization and Re-identification

Counterfactual Attention Learning Created by Yongming Rao*, Guangyi Chen*, Jiwen Lu, Jie Zhou This repository contains PyTorch implementation for ICCV

Yongming Rao 90 Dec 31, 2022
The code for MM2021 paper "Multi-Level Counterfactual Contrast for Visual Commonsense Reasoning"

The Code for MM2021 paper "Multi-Level Counterfactual Contrast for Visual Commonsense Reasoning" Setting up and using the repo Get the dataset. Follow

null 4 Apr 20, 2022
[CVPR 2021] Counterfactual VQA: A Cause-Effect Look at Language Bias

Counterfactual VQA (CF-VQA) This repository is the Pytorch implementation of our paper "Counterfactual VQA: A Cause-Effect Look at Language Bias" in C

Yulei Niu 94 Dec 3, 2022
code for CVPR paper Zero-shot Instance Segmentation

Code for CVPR2021 paper Zero-shot Instance Segmentation Code requirements python: python3.7 nvidia GPU pytorch1.1.0 GCC >=5.4 NCCL 2 the other python

zhengye 86 Dec 13, 2022
Open-Set Recognition: A Good Closed-Set Classifier is All You Need

Open-Set Recognition: A Good Closed-Set Classifier is All You Need Code for our paper: "Open-Set Recognition: A Good Closed-Set Classifier is All You

null 194 Jan 3, 2023
Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

Pytorch Implementation of Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic [Paper] [Colab is coming soon] Approach Example Usage To r

null 6 Dec 1, 2021
git git《Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking》(CVPR 2021) GitHub:git2] 《Masksembles for Uncertainty Estimation》(CVPR 2021) GitHub:git3]

Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li Accepted by CVPR

NingWang 236 Dec 22, 2022
Automatically download the cwru data set, and then divide it into training data set and test data set

Automatically download the cwru data set, and then divide it into training data set and test data set.自动下载cwru数据集，然后分训练数据集和测试数据集

null 6 Jun 27, 2022
Rethinking of Pedestrian Attribute Recognition: A Reliable Evaluation under Zero-Shot Pedestrian Identity Setting

Pytorch Pedestrian Attribute Recognition: A strong PyTorch baseline of pedestrian attribute recognition and multi-label classification.

Jian 79 Dec 18, 2022