This repository is for our paper Exploiting Scene Graphs for Human-Object Interaction Detection accepted by ICCV 2021.

Last update: Dec 20, 2022

Related tags

Deep Learning SG2HOI

Overview

SG2HOI

This repository is for our paper Exploiting Scene Graphs for Human-Object Interaction Detection accepted by ICCV 2021.

Installation

Pytorch 1.7.1

$ conda install --yes -c pytorch pytorch=1.7.1 torchvision cudatoolkit=11.0
$ pip install tdqm sklearn panda Pillow

maskrcnn

Check INSTALL.md to install maskrcnn. Then, adding the maskrcnn lib to your $PYTHONPATH, because our code uses the ROIAlign layer to extract the roi features.

Apex

If you want to use multiple gpus to train the model, you have to follow the instructions to install apex.

Datasets

HOI datasets

We use the off-the-shell object detection results of V-COCO and HICO from VSGnet, which can be downloaded from here.

Scene graph datasets

The scene graph prediction results are generated by TDE. Note that we use all the training and testing images of Visual Genome to train the SG model. Our pre-trained TDE model can be downloaded from here.

Training and testing

$ python main.py --gpu_id 0 --learning_rate 0.01 --batch_size 5 --num_epochs 50

Citations

If you find this project helps your research, please kindly consider citing our papers in your publications.

@InProceedings{he2021exploiting,
    author    = {He, Tao and Gao, Lianli and Song, Jingkuan and Li, Yuan-Fang},
    title     = {Exploiting Scene Graphs for Human-Object Interaction Detection},
    booktitle = {International Conference on Computer Vision(ICCV)},
    year      = {2021},
    url       = {https://arxiv.org/pdf/2108.08584}
}

Acknowledgement

This repository is developed on top of the other two projects: TDE by KaihuaTang and VSGnet by ASMIftekhar.

Comments

Excuse me, there are some problem when i try to run the code, and i need help

There is no 'main.py' in the code folder, and can only get the relecant content in 'train.py'. At the same time, there are no 'imports' and 'model_serialization' in the util folder. I'd like to ask for your help in this regard.

opened by charles-gong27 0
excuse me, can i ask a question about scene graph predicted results?

''' parser.add_argument('--sg_data', type=str, default="/mnt/hdd2/Object_Detections_vcoco/vcoco_", required=False,help="Your scene graph predicted results' path.") ''' can you provide the predicted results? Thank you!

opened by whyang78 0

Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts The rapid progress in 3D scene understanding has come with growing dem

182 Dec 30, 2022

Code for "ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on", accepted at WACV 2021 Generation of Human Behavior Workshop.

ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on [ Paper ] [ Project Page ] This repository contains the code fo

97 Dec 13, 2022

This is an official implementation for "Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation".

Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation This repo is the official implementation of Exploiting Temporal Con

241 Jan 7, 2023

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

MultiModal-InfoMax This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Informa

Deep Cognition and Language Research (DeCLaRe) Lab

89 Dec 26, 2022

[CVPR 2021] Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion

This repository is for our paper Exploiting Scene Graphs for Human-Object Interaction Detection accepted by ICCV 2021.

Related tags

Overview

SG2HOI

Installation

Pytorch 1.7.1

maskrcnn

Apex

Datasets

HOI datasets

Scene graph datasets

Training and testing

Citations

Acknowledgement

You might also like...

Code for CVPR 2021 oral paper "Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts"

Code for "ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on", accepted at WACV 2021 Generation of Human Behavior Workshop.

This is an official implementation for "Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation".

This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

[CVPR 2021] Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion

This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

[TIP 2020] Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion

Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch

Sync2Gen Code for ICCV 2021 paper: Scene Synthesis via Uncertainty-Driven Attribute Synchronization

Comments

Excuse me, there are some problem when i try to run the code, and i need help

excuse me, can i ask a question about scene graph predicted results?

Owner

HT

Repo for CVPR2021 paper "QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information"

Neural Scene Graphs for Dynamic Scene (CVPR 2021)

Populating 3D Scenes by Learning Human-Scene Interaction https://posa.is.tue.mpg.de/

Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

Code for our CVPR 2022 Paper "GEN-VLKT: Simplify Association and Enhance Interaction Understanding for HOI Detection"

Code for our NeurIPS 2021 paper 'Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation'

A weakly-supervised scene graph generation codebase. The implementation of our CVPR2021 paper ``Linguistic Structures as Weak Supervision for Visual Scene Graph Generation''

Pytorch implementation of Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors

An official implementation of "Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation" (ICCV 2021) in PyTorch.

This project is the official implementation of our accepted ICLR 2021 paper BiPointNet: Binary Neural Network for Point Clouds.