Official implementation of "Synthetic Temporal Anomaly Guided End-to-End Video Anomaly Detection" (ICCV Workshops 2021: RSL-CV).

Related tags

Deep Learning STEAL
Overview

Official PyTorch implementation of "Synthetic Temporal Anomaly Guided End-to-End Video Anomaly Detection"

This is the implementation of the paper "Synthetic Temporal Anomaly Guided End-to-End Video Anomaly Detection" (ICCV Workshops 2021: RSL-CV).

Paper || Presentation Video

Dependencies

  • Python 3.6
  • PyTorch = 1.7.0
  • Numpy
  • Sklearn

Datasets

Download the datasets into dataset folder, like ./dataset/ped2/, ./dataset/avenue/, ./dataset/shanghai/

Training

git clone https://github.com/aseuteurideu/STEAL
  • Training baseline
python train.py --dataset_type ped2
  • Training STEAL Net
python train.py --dataset_type ped2 --pseudo_anomaly_jump 0.2 --jump 2 3 4 5

Select --dataset_type from ped2, avenue, or shanghai.

For more details, check train.py

Pre-trained model

Model Dataset AUC Weight
Baseline Ped2 92.5% [ drive ]
Baseline Avenue 81.5% [ drive ]
Baseline ShanghaiTech 71.3% [ drive ]
STEAL Net Ped2 98.4% [ drive ]
STEAL Net Avenue 87.1% [ drive ]
STEAL Net ShanghaiTech 73.7% [ drive ]

Evaluation

  • Test the model
python evaluate.py --dataset_type ped2 --model_dir path_to_weight_file.pth
  • Test the model and save result image
python evaluate.py --dataset_type ped2 --model_dir path_to_weight_file.pth --img_dir folder_path_to_save_image_results
  • Test the model and generate demonstration video frames
python evaluate.py --dataset_type ped2 --model_dir path_to_weight_file.pth --vid_dir folder_path_to_save_video_results

Then compile the frames into video. For example, to compile the first video in ubuntu:

ffmpeg -framerate 10 -i frame_00_%04d.png -c:v libx264 -profile:v high -crf 20 -pix_fmt yuv420p video_00.mp4

Bibtex

@InProceedings{Astrid_2021_ICCV,
    author    = {Astrid, Marcella and Zaheer, Muhammad Zaigham and Lee, Seung-Ik},
    title     = {Synthetic Temporal Anomaly Guided End-to-End Video Anomaly Detection},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops},
    month     = {October},
    year      = {2021},
    pages     = {207-214}
}

Acknowledgement

The code is built on top of code provided by Park et al. [ github ] and Gong et al. [ github ]

Comments
  • About the parameter P

    About the parameter P

    hello,i have a quenstion:in paper,the p is 0.01,but in readme,p is 0.2,which one is true?moreover,i used 0.2 to train the code,Using my own training parameters and weights,the AUC of STEAL NET is 93.80%,why?

    opened by wangrui12345678 2
  • Dear author

    Dear author

    Hi, this is Dahee Han. Thanks for your paper and code.

    I'm writing to ask you about arguments in train.py. In your code, there are args.pseudo_anomaly_jump_inpainting at 42 and 54 lines. Does it mean args.pseudo_anomaly_jump?

    It would be very helpful if you check and answer it.

    opened by DaheeHan333 1
  • how to understand anomaly

    how to understand anomaly

    In your paper, you generate temporal pseudo anomalies by frame-skip. But why it can be abnormal?

    In normal status，the people appear and move in a slow speed, while they move faster in pseudo anomalies. Is this the starting point of generating temporal pseudo anomalies？

    Thank you！

    opened by wangherr 1
  • inconsistent numbers of samples

    inconsistent numbers of samples

    hello， when I use ped2 in Evaluate.py following line accuracy = AUC(anomaly_score_total_list, np.expand_dims(1-labels_list, 0)) causes ValueError: Found input variables with inconsistent numbers of samples: [1830, 1980]

    opened by herujin-jj 0
  • Error when trying to train STEAL model

    Error when trying to train STEAL model

    I was able to train the baseline model using the following command: python train.py --dataset_type ped2

    When I execute the following command: python train.py --dataset_type ped2 --pseudo_anomaly_jump 0.01 --jump 2 3 4 5

    I get the following error:

    rain.py --dataset_type ped2 --pseudo_anomaly_jump 0.01 --jump 2 3 4 5 exp_dir: logweight_recon_pajump0.01_jump[2,3,4,5] Traceback (most recent call last): File "train.py", line 115, in net_in[b] = imgsjump[0][b] IndexError: index 1 is out of bounds for dimension 0 with size 1

    Please advise

    opened by ToadTWP666 2
  • Model training

    Model training

    Hello, the default batchsize of two video cards can not achieve the expected effect. How many video cards does the author use for training? What is the batchsize?

    opened by 1079104484 2
Owner
Marcella Astrid
PhD candidate at University of Science and Technology, ETRI campus, South Korea
Marcella Astrid
GitHub
Official implementation of the paper Vision Transformer with Progressive Sampling, ICCV 2021.

Vision Transformer with Progressive Sampling This is the official implementation of the paper Vision Transformer with Progressive Sampling, ICCV 2021.

yuexy 123 Jan 1, 2023
Official implementation of the ICCV 2021 paper "Conditional DETR for Fast Training Convergence".

The DETR approach applies the transformer encoder and decoder architecture to object detection and achieves promising performance. In this paper, we handle the critical issue, slow training convergence, and present a conditional cross-attention mechanism for fast DETR training. Our approach is motivated by that the cross-attention in DETR relies highly on the content embeddings and that the spatial embeddings make minor contributions, increasing the need for high-quality content embeddings and thus increasing the training difficulty.

null 281 Dec 30, 2022
An official implementation of "Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation" (ICCV 2021) in PyTorch.

Exploiting a Joint Embedding Space for Generalized Zero-Shot Semantic Segmentation This is an official implementation of the paper "Exploiting a Joint

CV Lab @ Yonsei University 35 Oct 26, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page >> coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page >> coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022
The Official Implementation of the ICCV-2021 Paper: Semantically Coherent Out-of-Distribution Detection.

SCOOD-UDG (ICCV 2021) This repository is the official implementation of the paper: Semantically Coherent Out-of-Distribution Detection Jingkang Yang,

Jake YANG 62 Nov 21, 2022
[ICCV 2021] Official PyTorch implementation for Deep Relational Metric Learning.

Deep Relational Metric Learning This repository is the official PyTorch implementation of Deep Relational Metric Learning. Framework Datasets CUB-200-

Borui Zhang 39 Dec 10, 2022
Official implementation of the ICCV 2021 paper: "The Power of Points for Modeling Humans in Clothing".

The Power of Points for Modeling Humans in Clothing (ICCV 2021) This repository contains the official PyTorch implementation of the ICCV 2021 paper: T

Qianli Ma 158 Nov 24, 2022
official Pytorch implementation of ICCV 2021 paper FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting.

FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting By Rui Liu, Hanming Deng, Yangyi Huang, Xiaoyu Shi, Lewei Lu, Wenxiu

null 77 Dec 27, 2022
Official implementation of the ICCV 2021 paper "Joint Inductive and Transductive Learning for Video Object Segmentation"

JOINT This is the official implementation of Joint Inductive and Transductive learning for Video Object Segmentation, to appear in ICCV 2021. @inproce

Yunyao 35 Oct 16, 2022
[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

KPAC: Kernel-Sharing Parallel Atrous Convolutional block This repository contains the official Tensorflow implementation of the following paper: Singl

Hyeongseok Son 50 Dec 29, 2022
Official pytorch implementation of "Scaling-up Disentanglement for Image Translation", ICCV 2021.

Official pytorch implementation of "Scaling-up Disentanglement for Image Translation", ICCV 2021.

Aviv Gabbay 41 Nov 29, 2022
Official PyTorch Implementation of paper "Deep 3D Mask Volume for View Synthesis of Dynamic Scenes", ICCV 2021.

Deep 3D Mask Volume for View Synthesis of Dynamic Scenes Official PyTorch Implementation of paper "Deep 3D Mask Volume for View Synthesis of Dynamic S

Ken Lin 17 Oct 12, 2022
This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

?? ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

Hyungtae Lim 225 Dec 29, 2022
Official PyTorch implementation of N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras (ICCV 2021)

N-ImageNet: Towards Robust, Fine-Grained Object Recognition with Event Cameras Official PyTorch implementation of N-ImageNet: Towards Robust, Fine-Gra

null 32 Dec 26, 2022
Official Pytorch implementation of the paper "Action-Conditioned 3D Human Motion Synthesis with Transformer VAE", ICCV 2021

ACTOR Official Pytorch implementation of the paper "Action-Conditioned 3D Human Motion Synthesis with Transformer VAE", ICCV 2021. Please visit our we

Mathis Petrovich 248 Dec 23, 2022
Official implementation of Protected Attribute Suppression System, ICCV 2021

Official implementation of Protected Attribute Suppression System, ICCV 2021

Prithviraj Dhar 6 Jan 1, 2023
Official Repository for the ICCV 2021 paper "PixelSynth: Generating a 3D-Consistent Experience from a Single Image"

PixelSynth: Generating a 3D-Consistent Experience from a Single Image (ICCV 2021) Chris Rockwell, David F. Fouhey, and Justin Johnson [Project Website

Chris Rockwell 95 Nov 22, 2022
Official repository for "On Generating Transferable Targeted Perturbations" (ICCV 2021)

On Generating Transferable Targeted Perturbations (ICCV'21) Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli Paper:

Muzammal Naseer 46 Nov 17, 2022