Code for one-stage adaptive set-based HOI detector AS-Net.

Mingfei Chen

Last update: Dec 9, 2022

Related tags

Deep Learning AS-Net

Overview

AS-Net

Code for one-stage adaptive set-based HOI detector AS-Net.

Mingfei Chen*, Yue Liao*, Si Liu, Zhiyuan Chen, Fei Wang, Chen Qian. "Reformulating HOI Detection as Adaptive Set Prediction." Accepted to CVPR 2021. https://arxiv.org/abs/2103.05983

Installation

Environment

python >= 3.6

Install the dependencies.

 pip install -r requirements.txt

Data preparation

We first download the HICO-DET dataset.
The data should be prepared in the following structure:

data/hico
   |———  images
   |        └——————train
   |        |        └——————anno.json
   |        |        └——————XXX1.jpg
   |        |        └——————XXX2.jpg
   |        └——————test
   |                 └——————anno.json
   |                 └——————XXX1.jpg
   |                 └——————XXX2.jpg
   └——— test_hico.json
   └——— trainval_hico.json
   └——— rel_np.npy

Noted:

We transformed the original annotation files of HICO-DET to a *.json format, like data/hico/images/train_anno.json and ata/hico/images/test_hico.json.
test_hico.json, trainval_hico.json and rel_np.npy are used in the evaluation on HICO-DET. We provided these three files in our data/hico directory.
data/hico/train_anno.json and data/hico/images/train/anno.json are the same file. cp data/hico/train_anno.json data/hico/images/train/anno.json
data/hico/test_hico.json and data/hico/images/test/anno.json are the same file. cp data/hico/test_hico.json data/hico/images/test/anno.json

Evaluation

To evaluate our model on HICO-DET:

python3 tools/eval.py --cfg configs/hico.yaml MODEL.RESUME_PATH [checkpoint_path]

The checkpoint is saved on HICO-DET with torch==1.4.0.

Comments

a problem with the image shape of the input model

Hello Mingfei Chen, everything works normally when I use single GPU for training, but when I use multiple GPUs to train HOIA dataset, I encounter a problem, I would like to ask you how to solve it,there is a problem with the image shape of the input model

RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[8, 1, 864, 1251] to have 3 channels, but got 1 channels instead

opened by hutao568 7
The problem of result

Dear Chen, thanks for sharing this excellent idea, and I am very grateful for you. But there is a question I want to ask you is that I have found all the metrics is 0 after I run eval.py. Could you please tell the reason? Thank you very much!

opened by scxiaowu 7
Question about the evaluation code.

Thanks for your great work. There is a question haunting me about the evaluation process. The line 139 in hico_eval.py only ensures the human and the object are rightly localized. Why does the TP not consider the object class for a TP?

opened by JacobYuan7 4
The initialization of Qr and Qd ?

Dear Chen, thanks for sharing your excellent idea, after reading the paper, I have a question about Qr and Qd, are these two learnable variables randomly initialized?

opened by WXLL579 2

eval on hoia raise an ERROR

Hi, thanks for your work! When I eval on hoia dataset, I got this ERROR: bash eval_hoia.sh

python tools/eval.py --cfg configs/hoia.yaml \
    MODEL.RESUME_PATH checkpoints/ASNet_hoia_res50.pth

RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[1, 2, 800, 1066] to have 3 channels, but got 2 channels instead

But I print the img size at

def evaluate(self, eval_loader, mode, rel_topk=100):
        self.model.eval()
        results = []
        count = 0
        for data in tqdm(eval_loader):
            imgs, targets, filenames = data
            imgs = [img.to(self.device) for img in imgs]
            # targets are list type
            targets = [{k: v.to(self.device) for k, v in t.items()} for t in targets]
            bs = len(imgs)
            target_sizes = targets[0]['size'].expand(bs, 2)
            target_sizes = target_sizes.to(self.device)
            print("====================", imgs[0].size())
            outputs_dict = self.model(imgs)
            file_name = filenames[0]
            pred_out = self.postprocessors(outputs_dict, file_name, target_sizes,
                rel_topk=rel_topk)
            results.append(pred_out)
            count += 1

I got (3, 800, 1066). But in model forward, channel size changes to 2 ? Please help me~ I used ASNet_hoia_res50.pth on HOIA dataset

opened by zhaoxuyan 2

Resutls of Action recognition without object present in a scene

@yoyomimi hi thanks for sharing the code base great work, but i had one query, currently when i tested the model for some scenes like an only person running on a beach without any other object present there is no detections/activity in the output, is there any way i can get results like people walking , fighting, waving without depending on the object present in the scene

Thanks in advance

opened by abhigoku10 1

Owner

Mingfei Chen

You don't try, you will never know.

GitHub

Code for Mining the Benefits of Two-stage and One-stage HOI Detection

Status: Archive (code is provided as-is, no updates expected) PPO-EWMA [Paper] This is code for training agents using PPO-EWMA and PPG-EWMA, introduce

33 Dec 15, 2022

A light and fast one class detection framework for edge devices. We provide face detector, head detector, pedestrian detector, vehicle detector......

A Light and Fast Face Detector for Edge Devices Big News: LFD, which is a big update of LFFD, now is released (2021.03.09). It is strongly recommended

1.3k Dec 25, 2022

Code for Two-stage Identifier: "Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition"

Code for Two-stage Identifier: "Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition", accepted at ACL 2021. For details of the model and experiments, please see our paper.

87 Dec 16, 2022

This is the code for HOI Transformer

HOI Transformer Code for CVPR 2021 accepted paper End-to-End Human Object Interaction Detection with HOI Transformer. Reproduction We recomend you to

124 Dec 29, 2022

Code for our CVPR 2022 Paper "GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection"

GEN-VLKT Code for our CVPR 2022 paper "GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection". Contributed by Yue Lia

47 Dec 4, 2022

Virtual Dance Reality Stage: a feature that offers you to share a stage with another user virtually

Portrait Segmentation using Tensorflow This script removes the background from an input image. You can read more about segmentation here Setup The scr

291 Dec 24, 2022

Automatically download the cwru data set, and then divide it into training data set and test data set

Automatically download the cwru data set, and then divide it into training data set and test data set.自动下载cwru数据集，然后分训练数据集和测试数据集

6 Jun 27, 2022

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR2022)[paper] Authors: Chenhang He, Ruihuang Li, Shuai Li, L

141 Dec 30, 2022

A two-stage U-Net for high-fidelity denoising of historical recordings

A two-stage U-Net for high-fidelity denoising of historical recordings Official repository of the paper (not submitted yet): E. Moliner and V. Välimäk

57 Jan 5, 2023

A whale detector design for the Kaggle whale-detector challenge!

CNN (InceptionV1) + STFT based Whale Detection Algorithm So, this repository is my PyTorch solution for the Kaggle whale-detection challenge. The obje

92 Sep 28, 2021

Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Lane Follower This code is for the lane follower, including perception and control, as shown below. Environment Hardware Industrial Camera Intel-NUC(1

3 Jul 7, 2022

HeartRate detector with ArduinoandPython - Use Arduino and Python create a heartrate detector.

Syllabus of Contents Syllabus of Contents Introduction Of Project Features Develop With Python code introduction Installation License Developer Contac

1 Jan 5, 2022

Video lie detector using xgboost - A video lie detector using OpenFace and xgboost

video_lie_detector_using_xgboost a video lie detector using OpenFace and xgboost

2 Jan 11, 2022

Imposter-detector-2022 - HackED 2022 Team 3IQ - 2022 Imposter Detector

HackED 2022 Team 3IQ - 2022 Imposter Detector By Aneeljyot Alagh, Curtis Kan, Jo

3 Aug 20, 2022

(CVPR2021) DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation CVPR2021(oral) [arxiv] Requirements python3.7 pytorch==

85 Dec 7, 2022

[CVPR2021] Look before you leap: learning landmark features for one-stage visual grounding.

LBYL-Net This repo implements paper Look Before You Leap: Learning Landmark Features For One-Stage Visual Grounding CVPR 2021. Getting Started Prerequ

45 Dec 12, 2022

TOOD: Task-aligned One-stage Object Detection, ICCV2021 Oral

One-stage object detection is commonly implemented by optimizing two sub-tasks: object classification and localization, using heads with two parallel branches, which might lead to a certain level of spatial misalignment in predictions between the two tasks.

264 Jan 9, 2023

DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection

DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection Code for our Paper DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Obje

58 Dec 19, 2022

A Fast and Accurate One-Stage Approach to Visual Grounding, ICCV 2019 (Oral)

One-Stage Visual Grounding ***** New: Our recent work on One-stage VG is available at ReSC.***** A Fast and Accurate One-Stage Approach to Visual Grou

118 Dec 5, 2022

Code for one-stage adaptive set-based HOI detector AS-Net.

Related tags

Overview

AS-Net

Installation

Data preparation

Evaluation

Comments

a problem with the image shape of the input model

The problem of result

Question about the evaluation code.

The initialization of Qr and Qd ?

eval on hoia raise an ERROR

Resutls of Action recognition without object present in a scene

Owner

Mingfei Chen

Code for Mining the Benefits of Two-stage and One-stage HOI Detection

A light and fast one class detection framework for edge devices. We provide face detector, head detector, pedestrian detector, vehicle detector......

Code for Two-stage Identifier: "Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition"

This is the code for HOI Transformer

Code for our CVPR 2022 Paper "GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection"

Virtual Dance Reality Stage: a feature that offers you to share a stage with another user virtually

Automatically download the cwru data set, and then divide it into training data set and test data set

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

A two-stage U-Net for high-fidelity denoising of historical recordings

A whale detector design for the Kaggle whale-detector challenge!

Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

HeartRate detector with ArduinoandPython - Use Arduino and Python create a heartrate detector.

Video lie detector using xgboost - A video lie detector using OpenFace and xgboost

Imposter-detector-2022 - HackED 2022 Team 3IQ - 2022 Imposter Detector

(CVPR2021) DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

[CVPR2021] Look before you leap: learning landmark features for one-stage visual grounding.

TOOD: Task-aligned One-stage Object Detection, ICCV2021 Oral

DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection

A Fast and Accurate One-Stage Approach to Visual Grounding, ICCV 2019 (Oral)