Official implementation of "Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision" ECCV2020

Overview

XDVioDet

Official implementation of "Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision" ECCV2020.

The project website is XD-Violence. The features can be downloaded from our project website.

where we oversample each video frame with the “5-crop” augment, “5-crop” means cropping images into the center and four corners. _0.npy is the center, _1~ _4.npy is the corners.

How to train

  • download or extract the features.
  • use make_list.py in the list folder to generate the training and test list.
  • change the parameters in option.py
  • run main.py

How to test

  • run infer.py

      the model is in the ckpt folder.

Thanks for your attention!

Comments
  • Ground truth for training?

    Ground truth for training?

    I am trying to understand the code, and it seems like during training, the entire clip is labelled as violent or non-violent?

    However, during testing, there are test annotations for each frame. I was wondering if this is how the model works, or are there also training annotations being used during training?

    opened by Davidyao99 2
  • Some problems in the extracted features in training set

    Some problems in the extracted features in training set

    Interesting work! Recently, I would like to use the I3D extracted features to conduct some experiments on your dataset. But I found that some errors happened when loading the features from training set, such as:

    ValueError: cannot reshape array of size 6558518 into shape (6409,1024)

    So I wonder whether these feature files are cropped. I would appreciate it if you can check this issue. Thank you.

    opened by lts427 1
  • What exactly the metric AP means?

    What exactly the metric AP means?

    Hi, I found the metric mentioned in the paper is a bit confusing, you said corresponding area under the curve (average precision, AP), and in your code you used def precision_recall_curve(y_true, probas_pred, *, pos_label=None, sample_weight=None)to calculate precision and recall then further use auc to get pr_auc. What about average_precision_score, is the function the same as your AP calculation? Thanks

    opened by chengengliu 1
  • requirement.txt?

    requirement.txt?

    Hi, Thanks for generous sharing of your splendid work! Though the codes are available, I cannot find the requirement.txt or any dockerfile, which makes me difficult to build the same training environment(pytorch version, cuda version, etc) to reproduce or conduct further optimization. So I wonder if you could share the requirement.txt or provide the cuda/pytorch version of the training environment? Thanks!

    opened by JustinYuu 0
  • Frame level annotations for training data

    Frame level annotations for training data

    I am following the paper, and I was able to get the frame level annotations for the testing data, but not the training data. Is this file also made available in the resources? If so, where can I find it? Thanks.

    opened by dhruvapatil 1
  • Video cannot be played completely ?

    Video cannot be played completely ?

    Hello, I find some videoes in 1-1004.zip cannot be palyed completely by the local video player ? For an example, Mission.Impossible.II.2000__#01-49-35_01-51-52_label_B1-0-0.mp4 has more than 10 mins, but only about 2 mins can be played? What's wrong with this ?Can you give me some suggestions to deal with this problem ?

    opened by aries-young 0
  • 1005-2004.zip存在文件损坏,v=8cTqh9tMz_I__#1_label_A.mp4数据错误

    1005-2004.zip存在文件损坏,v=8cTqh9tMz_I__#1_label_A.mp4数据错误

    NewAdd.NBA-2017.12.25_CLE@GSW__#01-08-34_01-40-09_label_A.mp4 - CRC 校验错误。 Saving.Private.Ryan.1998__#02-29-31_02-30-55_label_B2-G-0.mp4 - CRC 校验错误。 v=8cTqh9tMz_I__#1_label_A.mp4 - 数据错误 - 该文件已损坏。 v=9eME1y6V-T4__#01-12-00_01-18-00_label_A.mp4 - CRC 校验错误。 出现错误

    已经重复下载过2次了,都报一样的错误。其中CRC校验错误的视频仍能播放,但是文件损坏的视频就无法播放了

    使用的软件是Bandizip,Win10平台

    其他压缩包没有问题

    opened by 137xiaoyu 3
  • Training videos from onedrive cannot be unzip

    Training videos from onedrive cannot be unzip

    I can't unzip 1-1004.zip and other zip files from trainning videos, which is download from ondrive.when I google it, I got one solution through jar -xvf 1-1004.zip,it can't work,and show this. Maybe someone else get the same problem,did it means there are something wrong with zip files java.util.zip.ZipException: only DEFLATED entries can have EXT descriptor at java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:310) at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:122) at sun.tools.jar.Main.extract(Main.java:979) at sun.tools.jar.Main.run(Main.java:311) at sun.tools.jar.Main.main(Main.java:1288)

    opened by masakirio 4
  • Audio feature extractor

    Audio feature extractor

    Hi, I am trying to extract audio feature on my own, but it does not reproduce the feature you gave. I am trying to run Vggish model, both implemented in tensorflow and pytorch, with any changes.

    Are there any changes from the original Vggish code when you extract the features? (ex. sampling rate, hop size... etc.)

    opened by yyyt1454 0
Owner
peng
PhD candidate of Xidian University
peng
Evaluation Pipeline for our ECCV2020: Journey Towards Tiny Perceptual Super-Resolution.

Journey Towards Tiny Perceptual Super-Resolution Test code for our ECCV2020 paper: https://arxiv.org/abs/2007.04356 Our x4 upscaling pre-trained model

Royson 6 Mar 30, 2022
PyTorch reimplementation of hand-biomechanical-constraints (ECCV2020)

Hand Biomechanical Constraints Pytorch Unofficial PyTorch reimplementation of Hand-Biomechanical-Constraints (ECCV2020). This project reimplement foll

Hao Meng 59 Dec 20, 2022
Code for "Learning Canonical Representations for Scene Graph to Image Generation", Herzig & Bar et al., ECCV2020

Learning Canonical Representations for Scene Graph to Image Generation (ECCV 2020) Roei Herzig*, Amir Bar*, Huijuan Xu, Gal Chechik, Trevor Darrell, A

roei_herzig 24 Jul 7, 2022
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Leo Xiao 3.9k Jan 5, 2023
Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

null 101 Nov 25, 2022
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021] Release Notes The offical PyTorch implementation of NeMo, p

Angtian Wang 76 Nov 23, 2022
StyleGAN2-ADA - Official PyTorch implementation

Abstract: Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes.

NVIDIA Research Projects 3.2k Dec 30, 2022
Official implementation of the ICLR 2021 paper

You Only Need Adversarial Supervision for Semantic Image Synthesis Official PyTorch implementation of the ICLR 2021 paper "You Only Need Adversarial S

Bosch Research 272 Dec 28, 2022
Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

This is the official PyTorch implementation of our paper: "Joint Object Detection and Multi-Object Tracking with Graph Neural Networks". Our project website and video demos are here.

Richard Wang 443 Dec 6, 2022
Official implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis https://arxiv.org/abs/2011.13775

CIPS -- Official Pytorch Implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis Requirements pip install -r requi

Multimodal Lab @ Samsung AI Center Moscow 201 Dec 21, 2022
Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

null 364 Dec 14, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022
Official PyTorch Implementation of Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity

UnRigidFlow This is the official PyTorch implementation of UnRigidFlow (IJCAI2019). Here are two sample results (~10MB gif for each) of our unsupervis

Liang Liu 28 Nov 16, 2022
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 35 Dec 6, 2022
Official implementation of Self-supervised Graph Attention Networks (SuperGAT), ICLR 2021.

SuperGAT Official implementation of Self-supervised Graph Attention Networks (SuperGAT). This model is presented at How to Find Your Friendly Neighbor

Dongkwan Kim 127 Dec 28, 2022
An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

PyTorch implementation of SFNet This is the implementation of the paper "SFNet: Learning Object-aware Semantic Correspondence". For more information,

CV Lab @ Yonsei University 87 Dec 30, 2022
This project is the official implementation of our accepted ICLR 2021 paper BiPointNet: Binary Neural Network for Point Clouds.

BiPointNet: Binary Neural Network for Point Clouds Created by Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Li

Haotong Qin 59 Dec 17, 2022
Official code implementation for "Personalized Federated Learning using Hypernetworks"

Personalized Federated Learning using Hypernetworks This is an official implementation of Personalized Federated Learning using Hypernetworks paper. [

Aviv Shamsian 121 Dec 25, 2022