Code for one-stage adaptive set-based HOI detector AS-Net.

Related tags

Deep Learning AS-Net
Overview

AS-Net

Code for one-stage adaptive set-based HOI detector AS-Net.

Mingfei Chen*, Yue Liao*, Si Liu, Zhiyuan Chen, Fei Wang, Chen Qian. "Reformulating HOI Detection as Adaptive Set Prediction." Accepted to CVPR 2021. https://arxiv.org/abs/2103.05983

Installation

Environment

  • python >= 3.6

Install the dependencies.

 pip install -r requirements.txt

Data preparation

  • We first download the HICO-DET dataset.
  • The data should be prepared in the following structure:
data/hico
   |———  images
   |        └——————train
   |        |        └——————anno.json
   |        |        └——————XXX1.jpg
   |        |        └——————XXX2.jpg
   |        └——————test
   |                 └——————anno.json
   |                 └——————XXX1.jpg
   |                 └——————XXX2.jpg
   └——— test_hico.json
   └——— trainval_hico.json
   └——— rel_np.npy

Noted:

  • We transformed the original annotation files of HICO-DET to a *.json format, like data/hico/images/train_anno.json and ata/hico/images/test_hico.json.
  • test_hico.json, trainval_hico.json and rel_np.npy are used in the evaluation on HICO-DET. We provided these three files in our data/hico directory.
  • data/hico/train_anno.json and data/hico/images/train/anno.json are the same file. cp data/hico/train_anno.json data/hico/images/train/anno.json
  • data/hico/test_hico.json and data/hico/images/test/anno.json are the same file. cp data/hico/test_hico.json data/hico/images/test/anno.json

Evaluation

To evaluate our model on HICO-DET:

python3 tools/eval.py --cfg configs/hico.yaml MODEL.RESUME_PATH [checkpoint_path]
  • The checkpoint is saved on HICO-DET with torch==1.4.0.
Comments
  • a problem with the image shape of the input model

    a problem with the image shape of the input model

    Hello Mingfei Chen, everything works normally when I use single GPU for training, but when I use multiple GPUs to train HOIA dataset, I encounter a problem, I would like to ask you how to solve it,there is a problem with the image shape of the input model

    RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[8, 1, 864, 1251] to have 3 channels, but got 1 channels instead

    opened by hutao568 7
  • The problem of result

    The problem of result

    Dear Chen, thanks for sharing this excellent idea, and I am very grateful for you. But there is a question I want to ask you is that I have found all the metrics is 0 after I run eval.py. Could you please tell the reason? Thank you very much!

    opened by scxiaowu 7
  • Question about the evaluation code.

    Question about the evaluation code.

    Thanks for your great work. There is a question haunting me about the evaluation process. The line 139 in hico_eval.py only ensures the human and the object are rightly localized. Why does the TP not consider the object class for a TP?

    opened by JacobYuan7 4
  • The initialization of Qr and Qd ?

    The initialization of Qr and Qd ?

    Dear Chen, thanks for sharing your excellent idea, after reading the paper, I have a question about Qr and Qd, are these two learnable variables randomly initialized?

    opened by WXLL579 2
  • eval on hoia raise an ERROR

    eval on hoia raise an ERROR

    Hi, thanks for your work! When I eval on hoia dataset, I got this ERROR: bash eval_hoia.sh

    python tools/eval.py --cfg configs/hoia.yaml \
    MODEL.RESUME_PATH checkpoints/ASNet_hoia_res50.pth

    RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[1, 2, 800, 1066] to have 3 channels, but got 2 channels instead

    But I print the img size at

    def evaluate(self, eval_loader, mode, rel_topk=100):
        self.model.eval()
        results = []
        count = 0
        for data in tqdm(eval_loader):
            imgs, targets, filenames = data
            imgs = [img.to(self.device) for img in imgs]
            # targets are list type
            targets = [{k: v.to(self.device) for k, v in t.items()} for t in targets]
            bs = len(imgs)
            target_sizes = targets[0]['size'].expand(bs, 2)
            target_sizes = target_sizes.to(self.device)
            print("====================", imgs[0].size())
            outputs_dict = self.model(imgs)
            file_name = filenames[0]
            pred_out = self.postprocessors(outputs_dict, file_name, target_sizes,
                rel_topk=rel_topk)
            results.append(pred_out)
            count += 1

    I got (3, 800, 1066). But in model forward, channel size changes to 2 ? Please help me~ I used ASNet_hoia_res50.pth on HOIA dataset

    opened by zhaoxuyan 2
  • Resutls of Action recognition without object present in a scene

    Resutls of Action recognition without object present in a scene

    @yoyomimi hi thanks for sharing the code base great work, but i had one query, currently when i tested the model for some scenes like an only person running on a beach without any other object present there is no detections/activity in the output, is there any way i can get results like people walking , fighting, waving without depending on the object present in the scene

    Thanks in advance

    opened by abhigoku10 1
Owner
Mingfei Chen
You don't try, you will never know.
Mingfei Chen
GitHub
Code for Mining the Benefits of Two-stage and One-stage HOI Detection

Status: Archive (code is provided as-is, no updates expected) PPO-EWMA [Paper] This is code for training agents using PPO-EWMA and PPG-EWMA, introduce

OpenAI 33 Dec 15, 2022
A light and fast one class detection framework for edge devices. We provide face detector, head detector, pedestrian detector, vehicle detector......

A Light and Fast Face Detector for Edge Devices Big News: LFD, which is a big update of LFFD, now is released (2021.03.09). It is strongly recommended

YonghaoHe 1.3k Dec 25, 2022
Code for Two-stage Identifier: "Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition"

Code for Two-stage Identifier: "Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition", accepted at ACL 2021. For details of the model and experiments, please see our paper.

tricktreat 87 Dec 16, 2022
This is the code for HOI Transformer

HOI Transformer Code for CVPR 2021 accepted paper End-to-End Human Object Interaction Detection with HOI Transformer. Reproduction We recomend you to

BigBangEpoch 124 Dec 29, 2022
Code for our CVPR 2022 Paper "GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection"

GEN-VLKT Code for our CVPR 2022 paper "GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection". Contributed by Yue Lia

Yue Liao 47 Dec 4, 2022
Virtual Dance Reality Stage: a feature that offers you to share a stage with another user virtually

Portrait Segmentation using Tensorflow This script removes the background from an input image. You can read more about segmentation here Setup The scr

null 291 Dec 24, 2022
Automatically download the cwru data set, and then divide it into training data set and test data set

Automatically download the cwru data set, and then divide it into training data set and test data set.自动下载cwru数据集，然后分训练数据集和测试数据集

null 6 Jun 27, 2022
Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR 2022)

Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds (CVPR2022)[paper] Authors: Chenhang He, Ruihuang Li, Shuai Li, L

Billy HE 141 Dec 30, 2022
A two-stage U-Net for high-fidelity denoising of historical recordings

A two-stage U-Net for high-fidelity denoising of historical recordings Official repository of the paper (not submitted yet): E. Moliner and V. Välimäk

Eloi Moliner Juanpere 57 Jan 5, 2023
A whale detector design for the Kaggle whale-detector challenge!

CNN (InceptionV1) + STFT based Whale Detection Algorithm So, this repository is my PyTorch solution for the Kaggle whale-detection challenge. The obje

Tarin Ziyaee 92 Sep 28, 2021
Lane follower: Lane-detector (OpenCV) + Object-detector (YOLO5) + CAN-bus

Lane Follower This code is for the lane follower, including perception and control, as shown below. Environment Hardware Industrial Camera Intel-NUC(1

Siqi Fan 3 Jul 7, 2022
HeartRate detector with ArduinoandPython - Use Arduino and Python create a heartrate detector.

Syllabus of Contents Syllabus of Contents Introduction Of Project Features Develop With Python code introduction Installation License Developer Contac

null 1 Jan 5, 2022
Video lie detector using xgboost - A video lie detector using OpenFace and xgboost

video_lie_detector_using_xgboost a video lie detector using OpenFace and xgboost

null 2 Jan 11, 2022
Imposter-detector-2022 - HackED 2022 Team 3IQ - 2022 Imposter Detector

HackED 2022 Team 3IQ - 2022 Imposter Detector By Aneeljyot Alagh, Curtis Kan, Jo

Joshua Ji 3 Aug 20, 2022
(CVPR2021) DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation CVPR2021(oral) [arxiv] Requirements python3.7 pytorch==

W-zx-Y 85 Dec 7, 2022
[CVPR2021] Look before you leap: learning landmark features for one-stage visual grounding.

LBYL-Net This repo implements paper Look Before You Leap: Learning Landmark Features For One-Stage Visual Grounding CVPR 2021. Getting Started Prerequ

SVIP Lab 45 Dec 12, 2022
TOOD: Task-aligned One-stage Object Detection, ICCV2021 Oral

One-stage object detection is commonly implemented by optimizing two sub-tasks: object classification and localization, using heads with two parallel branches, which might lead to a certain level of spatial misalignment in predictions between the two tasks.

null 264 Jan 9, 2023
DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection

DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Object Detection Code for our Paper DAFNe: A One-Stage Anchor-Free Deep Model for Oriented Obje

Steven Lang 58 Dec 19, 2022
A Fast and Accurate One-Stage Approach to Visual Grounding, ICCV 2019 (Oral)

One-Stage Visual Grounding ***** New: Our recent work on One-stage VG is available at ReSC.***** A Fast and Accurate One-Stage Approach to Visual Grou

Zhengyuan Yang 118 Dec 5, 2022