Normal Learning in Videos with Attention Prototype Network

Last update: Dec 13, 2022

Related tags

Deep Learning APN

Overview

Codes_APN

Official codes of CVPR21 paper: Normal Learning in Videos with Attention Prototype Network (https://arxiv.org/abs/2108.11055)

Overview of our approach based on APU and CAU model:

Introduction

Frame reconstruction (current or future frame) based on Auto-Encoder (AE) is a popular method for video anomaly detection. With models trained on the normal data, the reconstruction errors of anomalous scenes are usually much larger than those of normal ones. Previous methods introduced the memory bank into AE, for encoding diverse normal patterns across the training videos. However, they are memory consuming and cannot cope with unseen new scenarios in the testing data. In this work, we propose a self-attention prototype unit (APU) to encode the normal latent space as prototypes in real time, free from extra memory cost. In addition, we introduce circulative attention mechanism to our backbone to form a novel feature extracting learner, namely Circulative Attention Unit(CAU). It enables the fast adaption capability on new scenes by only consuming a few iterations of update. Extensive experiments are conducted on various benchmarks. The superior performance over the state-of-the-art demonstrates the effectiveness of our method.

Performance

We achieved SOTA on many video anomaly detection datasets.

Unsupervised Anomaly Detection Model Training

bash train.sh

Unsupervised Anomaly Detection Model Testing

bash test.sh

If you find this work helpful, please cite:

@inproceedings{Nv2021APN,
  author    = {Chao Hu and
	       Fan Wu and
               Weijie Wu and
               Weibin Qiu and
               Shengxin Lai},
  title     = {Normal Learning in Videos with Attention Prototype Network},
  booktitle = {Computer Vision and Pattern Recognition},
  year      = {2021}
}

Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

CoProtector Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

1 Oct 26, 2021

Prototype python implementation of the ome-ngff table spec

8 Nov 20, 2022

[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

96 Dec 30, 2022

CVPR2022 (Oral) - Rethinking Semantic Segmentation: A Prototype View

Rethinking Semantic Segmentation: A Prototype View Rethinking Semantic Segmentation: A Prototype View, Tianfei Zhou, Wenguan Wang, Ender Konukoglu and

239 Dec 26, 2022

PyKale is a PyTorch library for multimodal learning and transfer learning as well as deep learning and dimensionality reduction on graphs, images, texts, and videos

PyKale is a PyTorch library for multimodal learning and transfer learning as well as deep learning and dimensionality reduction on graphs, images, texts, and videos. By adopting a unified pipeline-based API design, PyKale enforces standardization and minimalism, via reusing existing resources, reducing repetitions and redundancy, and recycling learning models across areas.

370 Dec 27, 2022

Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Transformer in Transformer Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image c

272 Dec 23, 2022

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

HaloNet - Pytorch Implementation of the Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones. This re

189 Nov 22, 2022

Implementation of STAM (Space Time Attention Model), a pure and simple attention model that reaches SOTA for video classification

STAM - Pytorch Implementation of STAM (Space Time Attention Model), yet another pure and simple SOTA attention model that bests all previous models in

109 Dec 28, 2022

Attention-driven Robot Manipulation (ARM) which includes Q-attention

Attention-driven Robotic Manipulation (ARM) This codebase is home to: Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation I

84 Dec 29, 2022

Comments

Trained Model for Shanghai dataset?

Hello,

It seems a very interesting work. I am able to reproduce results for Ped2 dataset but not for Shanghai dataset (I got only 74.7% AUC). Can you provide your trained model so that I can verify the results? Also, any tips on why I cannot reproduce the results.

Thanks in advance

opened by devashishlohani 0
What's the difference between CAU and Criss-Cross Attention?

Hello! It's a nice work! APU shows really significant improvements on diverse VAD datasets. However, I have a question. What's the difference between CAU and Criss-Cross Attention [1], please? The paper does not cite CCA [1].

[1] Huang Z, Wang X, Huang L, et al. Ccnet: Criss-cross attention for semantic segmentation[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 603-612.

opened by zugexiaodui 0

Normal Learning in Videos with Attention Prototype Network

Related tags

Overview

Codes_APN

Introduction

Performance

Unsupervised Anomaly Detection Model Training

Unsupervised Anomaly Detection Model Testing

You might also like...

Code for the prototype tool in our paper "CoProtector: Protect Open-Source Code against Unauthorized Training Usage with Data Poisoning".

Prototype python implementation of the ome-ngff table spec

[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

CVPR2022 (Oral) - Rethinking Semantic Segmentation: A Prototype View

PyKale is a PyTorch library for multimodal learning and transfer learning as well as deep learning and dimensionality reduction on graphs, images, texts, and videos

Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

Implementation of STAM (Space Time Attention Model), a pure and simple attention model that reaches SOTA for video classification

Attention-driven Robot Manipulation (ARM) which includes Q-attention

Comments

Trained Model for Shanghai dataset?

What's the difference between CAU and Criss-Cross Attention?

Owner

Code Release for ICCV 2021 (oral), "AdaFit: Rethinking Learning-based Normal Estimation on Point Clouds"

Adaptive Prototype Learning and Allocation for Few-Shot Segmentation (CVPR 2021)

Implementation of the paper "Self-Promoted Prototype Refinement for Few-Shot Class-Incremental Learning"

Audio-Visual Generalized Few-Shot Learning with Prototype-Based Co-Adaptation

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation

An implementation of a discriminant function over a normal distribution to help classify datasets.

A simple rest api that classifies pneumonia infection weather it is Normal, Pneumonia Virus or Pneumonia Bacteria from a chest-x-ray image.

Prototype for Baby Action Detection and Classification

functorch is a prototype of JAX-like composable function transforms for PyTorch.