Rethinking Transformer-based Set Prediction for Object Detection

Zhiqing Sun

Last update: Dec 3, 2022

Related tags

Deep Learning TSP-Detection

Overview

Rethinking Transformer-based Set Prediction for Object Detection

Here are the code for the ICCV paper. The code is adapted from Detectron2 and AdelaiDet.

All the model are trained on 4 V100 GPUs.

Prerequisites

Modify the environment name and environment prefix in environment.yml and run

conda env create -f environment.yml

git clone https://github.com/facebookresearch/detectron2.git
cd detectron2
git reset --hard b88c6c06563e4db1139aafbd6d8d97d1fa7a57e4
pip install -e .

Rreproducing Results

For TSP-FCOS,

bash tsp_fcos.sh

For TSP-RCNN,

bash tsp_rcnn.sh

Citation

@InProceedings{Sun_2021_ICCV,
    author    = {Sun, Zhiqing and Cao, Shengcao and Yang, Yiming and Kitani, Kris M.},
    title     = {Rethinking Transformer-Based Set Prediction for Object Detection},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {3611-3620}
}

You might also like...

Tutorial to set up TensorFlow Object Detection API on the Raspberry Pi

A tutorial showing how to set up TensorFlow's Object Detection API on the Raspberry Pi

1.1k Dec 26, 2022

Official code of the paper "Expanding Low-Density Latent Regions for Open-Set Object Detection" (CVPR 2022)

OpenDet Expanding Low-Density Latent Regions for Open-Set Object Detection (CVPR2022) Jiaming Han, Yuqiang Ren, Jian Ding, Xingjia Pan, Ke Yan, Gui-So

64 Jan 7, 2023

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

VSR-Transformer By Jiezhang Cao, Yawei Li, Kai Zhang, Luc Van Gool This paper proposes a new Transformer for video super-resolution (called VSR-Transf

225 Nov 13, 2022

Code Release for ICCV 2021 (oral), "AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds"

AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds (ICCV 2021 oral) **Project Page | Arxiv ** Runsong Zhu¹, Yuan Liu², Zhen Dong¹, Te

40 Dec 30, 2022

This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

This is the repository for our 2020 paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis". Data We provide

35 Nov 16, 2022

Open-Set Recognition: A Good Closed-Set Classifier is All You Need

Open-Set Recognition: A Good Closed-Set Classifier is All You Need Code for our paper: "Open-Set Recognition: A Good Closed-Set Classifier is All You

194 Jan 3, 2023

Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of images as "pixels"

picinpics Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of

1 Oct 24, 2021

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

Episodic Transformers (E.T.) Episodic Transformer for Vision-and-Language Navigation Alexander Pashevich, Cordelia Schmid, Chen Sun Episodic Transform

62 Dec 24, 2022

Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera.

Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera. This project prepares training and testing data for various deep learning projects such as 6D object pose estimation projects singleshotpose, as well as object detection and instance segmentation projects.

305 Dec 16, 2022

Comments

Run the error in the trin_net_rcnn.py

Hello, I encountered errors in the picture during the process of running the program. Excuse me, is Troi in the picture a module in Python? But after installing the error in the picture, I hope you can help to point out.

opened by 7linlin 2
how to train/infer with encoder-only experiment?

if I understand correctly, without Decoder, you would endup with thousands of proposals from FPN, how to train such MultiHeadAttention?

Also, correct me if I am wrong. Your work is pretty much a reformulation of RelationNet, which using attention to reduce FP after standard detection pipeline. Therefore convergence is no longer a issue, but still relies on NMS and anchor/point.

opened by qianyizhang 1

Owner

Zhiqing Sun

Third-year Ph.D. student at LTI, CMU

GitHub

The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer"

Shuffle Transformer The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer" Introduction Very recently, window-

87 Nov 29, 2022

TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios

TPH-YOLOv5 This repo is the implementation of "TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured

439 Dec 22, 2022

GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification

GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification This is the official pytorch implementation of t

5 Nov 14, 2022

Automatically download the cwru data set, and then divide it into training data set and test data set

Automatically download the cwru data set, and then divide it into training data set and test data set.自动下载cwru数据集，然后分训练数据集和测试数据集

6 Jun 27, 2022

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Swin Transformer for Object Detection This repo contains the supported code and configuration files to reproduce object detection results of Swin Tran

1.4k Dec 30, 2022

MOT-Tracking-by-Detection-Pipeline - For Tracking-by-Detection format MOT (Multi Object Tracking), is it a framework that separates Detection and Tracking processes?

MOT-Tracking-by-Detection-Pipeline Tracking-by-Detection形式のMOT(Multi Object Trac

41 Nov 23, 2022

Price-Prediction-For-a-Dream-Home - A machine learning based linear regression trained model for house price prediction.

Price-Prediction-For-a-Dream-Home ROADMAP TO THIS LINEAR REGRESSION BASED HOUSE PRICE PREDICTION PREDICTION MODEL Import all the dependencies of the p

1 Dec 29, 2021

Rethinking Transformer-based Set Prediction for Object Detection

Related tags

Overview

Rethinking Transformer-based Set Prediction for Object Detection

Prerequisites

Rreproducing Results

Citation

You might also like...

Tutorial to set up TensorFlow Object Detection API on the Raspberry Pi

Official code of the paper "Expanding Low-Density Latent Regions for Open-Set Object Detection" (CVPR 2022)

VSR-Transformer - This paper proposes a new Transformer for video super-resolution (called VSR-Transformer).

Code Release for ICCV 2021 (oral), "AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds"

This is our ARTS test set, an enriched test set to probe Aspect Robustness of ABSA.

Open-Set Recognition: A Good Closed-Set Classifier is All You Need

Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of images as "pixels"

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera.

Comments

Run the error in the trin_net_rcnn.py

how to train/infer with encoder-only experiment?

Owner

Zhiqing Sun

The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer"

TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios

GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification

Automatically download the cwru data set, and then divide it into training data set and test data set

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation

Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

Yolo object detection - Yolo object detection with python

MOT-Tracking-by-Detection-Pipeline - For Tracking-by-Detection format MOT (Multi Object Tracking), is it a framework that separates Detection and Tracking processes?

Price-Prediction-For-a-Dream-Home - A machine learning based linear regression trained model for house price prediction.