General Multi-label Image Classification with Transformers

Related tags

Deep Learning computer-vision transformers multi-label-classification
Overview

General Multi-label Image Classification with Transformers
Jack Lanchantin, Tianlu Wang, Vicente Ordóñez Román, Yanjun Qi
Conference on Computer Vision and Pattern Recognition (CVPR) 2021
[paper] [poster] [slides]

Training and Running C-Tran

Python version 3.7 is required and all major packages used and their versions are listed in requirements.txt.

C-Tran on COCO80 Dataset

Download COCO data (19G)

wget http://cs.virginia.edu/~jjl5sw/data/vision/coco.tar.gz
mkdir -p data/
tar -xvf coco.tar.gz -C data/

Train New Model

python main.py  --batch_size 16  --lr 0.00001 --optim 'adam' --layers 3  --dataset 'coco' --use_lmt --dataroot data/

C-Tran on VOC20 Dataset

Download VOC2007 data (1.7G)

wget http://cs.virginia.edu/~jjl5sw/data/vision/voc.tar.gz
mkdir -p data/
tar -xvf voc.tar.gz -C data/

Train New Model

python main.py  --batch_size 16  --lr 0.00001 --optim 'adam' --layers 3  --dataset 'voc' --use_lmt --grad_ac_step 2 --dataroot data/

Citing

@article{lanchantin2020general,
  title={General Multi-label Image Classification with Transformers},
  author={Lanchantin, Jack and Wang, Tianlu and Ordonez, Vicente and Qi, Yanjun},
  journal={arXiv preprint arXiv:2011.14027},
  year={2020}
}
Comments
  • A question of the args

    A question of the args "--grad_ac_step"

    Dear author,

    Thanks very much for your interesting work! I wonder why the args "--grad_ac_step" is set as 2 for VOC2007?

    From your code, I see that if you set "--grad_ac_step" as 2, a minibatch images (16 images) will not be used for optimization. Can you help me solve this issue?

    Thanks very much!

    opened by Kyfafyd 12
  • About applications on custom datasets

    About applications on custom datasets

    Thank you very much for your work, but I am confused about some code of your dataset, especially the coco type. Your code is not regular COCO dataset, and you did not provide relevant code to convert to your format, which made it very difficult for me to migrate to my own dataset.

    opened by Breeze-Zero 3
  • Weird same prediction with large dataset

    Weird same prediction with large dataset

    Thank you for your wonderful project!

    I have been doing research with your C-Tran project, and conducting training with several autonomous driving dataset, such as nuScenes and METEOR.

    It seems that when the number of images is larger than 100,000, all of the images tend to have almost same predictions. For example, [car: 0.96, pedestrian: 0.01, truck: 0.48] for all of the images, even though labels of different images are different. However, when I shrink the number of images to 3000 or less, different images will have different predictions of labels.

    May I ask why this problem happens? Is it because I failed to apply your network correctly, or because the network with Transformers struggled to output satisfying predictions with large data?

    Thank you for your reading, and I look forward to your reply.

    opened by shinu335 2
  • ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm)

    ERROR: Unexpected bus error encountered in worker. This might be caused by insufficient shared memory (shm)

    p-241122-1 I tried running your codes on JupyterLab, but there is a persistent error popping up every time I run it according to the steps you mentioned in README.md, and I'm unable to solve it. I have attached a screenshot of the same. I would be obliged if you could help me understand or solve the problem.

    opened by sumanchowdhurykgp 1
  • A question about argument

    A question about argument "--pos_emb" and a possible comparsive method.

    Dear author,

    Thanks for your time!

    I have noticed that their is an argument "--pos_emb", so how to use it, If it will improve performance? I found if I call it when training, it will raise an error.

    Meanwhile, I wonder if this AAAI2021 paper can be compared with your paper? It seems these two papers are using the same setting. https://ojs.aaai.org/index.php/AAAI/article/view/17098/16905

    opened by Kyfafyd 1
  • coco datasets

    coco datasets

    Can you provide the content in "train.data" and "val_test.data" dirs? I have the coco images and annotations already according to the original coco website.

    opened by valencebond 1
  • About applications on custom datasets

    About applications on custom datasets

    Thank you very much for your work, but I am confused about some code of your dataset, especially the coco type. Your code is not regular COCO dataset, and you did not provide relevant code to convert to your format, which made it very difficult for me to migrate to my own dataset.

    opened by Breeze-Zero 0
  • Hoping for your code!

    Hoping for your code!

    Thank you for your exciting work, I still have some detail implementation problems and would like to try some more ideas. So hoping for your code! (●'◡'●)

    opened by woxue 0
  • Optimal no. of epochs to train a model and availability of a trained model

    Optimal no. of epochs to train a model and availability of a trained model

    Hi, Thank you for the code and the great work. I have two questions regarding your work. First how many number of epochs your model needs to train successfully? I am running your model on my dataset however it is not learning till 10 epochs therefore I want to know how many epochs your model needs. Secondly do you also have trained models as transformers need a lot of resources for training. Therefore if you can provide trained models that would help me a lot. Thank you for your time.

    opened by ans92 1
  • Running

    Running

    Excuse me, I am very interested in your work. But the End2EndModel method is not found in this project. And MAP didn't live up to expectations. I'm running on Windoes Thank you very much

    opened by CQYIO 0
Owner
QData
http://www.cs.virginia.edu/yanjun/
QData
GitHub
Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

U2Fusion Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal (VIS-IR, medical), multi

Han Xu 129 Dec 11, 2022
《LightXML: Transformer with dynamic negative sampling for High-Performance Extreme Multi-label Text Classification》(AAAI 2021) GitHub:

LightXML: Transformer with dynamic negative sampling for High-Performance Extreme Multi-label Text Classification

null 76 Dec 5, 2022
Official Pytorch Implementation of: "Semantic Diversity Learning for Zero-Shot Multi-label Classification"(2021) paper

Semantic Diversity Learning for Zero-Shot Multi-label Classification Paper Official PyTorch Implementation Avi Ben-Cohen, Nadav Zamir, Emanuel Ben Bar

null 28 Aug 29, 2022
Official implementation of paper "Query2Label: A Simple Transformer Way to Multi-Label Classification".

Introdunction This is the official implementation of the paper "Query2Label: A Simple Transformer Way to Multi-Label Classification". Abstract This pa

Shilong Liu 274 Dec 28, 2022
Official implementation for the paper: "Multi-label Classification with Partial Annotations using Class-aware Selective Loss"

Multi-label Classification with Partial Annotations using Class-aware Selective Loss Paper | Pretrained models Official PyTorch Implementation Emanuel

null 99 Dec 27, 2022
PyTorch implementation of Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network

hierarchical-multi-label-text-classification-pytorch Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network Approach This

Mingu Kang 17 Dec 13, 2022
A benchmark dataset for mesh multi-label-classification based on cube engravings introduced in MeshCNN

Double Cube Engravings This script creates a dataset for multi-label mesh clasification, with an intentionally difficult setup for point cloud classif

Yotam Erel 1 Nov 30, 2021
This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

This project aim to create multi-label classification annotation tool to boost annotation speed and make it more easier.

null 4 Aug 2, 2022
Official implementation of "Open-set Label Noise Can Improve Robustness Against Inherent Label Noise" (NeurIPS 2021)

Open-set Label Noise Can Improve Robustness Against Inherent Label Noise NeurIPS 2021: This repository is the official implementation of ODNL. Require

Hongxin Wei 12 Dec 7, 2022
A PyTorch implementation of ICLR 2022 Oral paper PiCO: Contrastive Label Disambiguation for Partial Label Learning

PiCO: Contrastive Label Disambiguation for Partial Label Learning This is a PyTorch implementation of ICLR 2022 Oral paper PiCO; also see our Project

王皓波 83 May 11, 2022
Simple-Image-Classification - Simple Image Classification Code (PyTorch)

Simple-Image-Classification Simple Image Classification Code (PyTorch) Yechan Kim This repository contains: Python3 / Pytorch code for multi-class ima

Yechan Kim 8 Oct 29, 2022
Image Classification - A research on image classification and auto insurance claim prediction, a systematic experiments on modeling techniques and approaches

A research on image classification and auto insurance claim prediction, a systematic experiments on modeling techniques and approaches

null 0 Jan 23, 2022
ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection

ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection This repository contains implementation of the

Visual Understanding Lab @ Samsung AI Center Moscow 190 Dec 30, 2022
Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

null 101 Nov 25, 2022
DECAF: Deep Extreme Classification with Label Features

DECAF DECAF: Deep Extreme Classification with Label Features @InProceedings{Mittal21, author = "Mittal, A. and Dahiya, K. and Agrawal, S. and Sain

null 46 Nov 6, 2022
Official implementation of "SinIR: Efficient General Image Manipulation with Single Image Reconstruction" (ICML 2021)

SinIR (Official Implementation) Requirements To install requirements: pip install -r requirements.txt We used Python 3.7.4 and f-strings which are in

null 47 Oct 11, 2022
Spectralformer: Rethinking hyperspectral image classification with transformers

The code in this toolbox implements the "Spectralformer: Rethinking hyperspectral image classification with transformers". More specifically, it is detailed as follow.

Danfeng Hong 104 Jan 4, 2023
Shared Attention for Multi-label Zero-shot Learning

Shared Attention for Multi-label Zero-shot Learning Overview This repository contains the implementation of Shared Attention for Multi-label Zero-shot

dathuynh 26 Dec 14, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page >> coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022