Yolov5+SlowFast: Realtime Action Detection Based on PytorchVideo

Related tags

Deep Learning yolo_slowfast
Overview

Yolov5+SlowFast: Realtime Action Detection

A realtime action detection frame work based on PytorchVideo.

Here are some details about our modification:

  • we choose yolov5 as an object detector instead of detectron2, it is faster and more convenient
  • we use a tracker(deepsort) to allocate action labels to all objects(with same ids) in different frames
  • our processing speed reached 24.2 FPS at 30 inference barch size (on a single RTX 2080Ti GPU)

Relevant infomation: FAIR/PytorchVideo; Ultralytics/Yolov5

Demo comparison betwween original(<-left) and ours(->right).

Installation

  1. create a new python environment:

    conda create -n env_name python=3.7.11

  2. install requiments:

    pip install -r requirements.txt

  3. download weights file(ckpt.t7) from [deepsort] to this folder:

    ./deep_sort/deep_sort/deep/checkpoint/

  4. test on your video:

    python yolo_slowfast.py --input {path to your video}

    The first time to execute this command may take some times to download the yolov5 code and it's weights file from torch.hub, keep your network connected.

References

Thanks for these great works:

[1] Ultralytics/Yolov5

[2] ZQPei/deepsort

[3] FAIR/PytorchVideo

[2] AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions. paper

[3] SlowFast Networks for Video Recognition. paper

Citation

If you find our work useful, please cite as follow:

{   yolo_slowfast,
    author = {Wu Fan},
    title = { A realtime action detection frame work based on PytorchVideo},
    year = {2021},
    url = {\url{https://github.com/wufan-tb/gmm_dae}}
}
You might also like...
joint detection and semantic segmentation, based on ultralytics/yolov5,
joint detection and semantic segmentation, based on ultralytics/yolov5,

Multi YOLO V5——Detection and Semantic Segmentation Overeview This is my undergraduate graduation project which based on ultralytics YOLO V5 tag v5.0.

null 477 Jan 6, 2023
Frigate - NVR With Realtime Object Detection for IP Cameras
Frigate - NVR With Realtime Object Detection for IP Cameras

A complete and local NVR designed for HomeAssistant with AI object detection. Uses OpenCV and Tensorflow to perform realtime object detection locally for IP cameras.

Blake Blackshear 6.4k Dec 31, 2022
Realtime YOLO Monster Detection With Non Maximum Supression
Realtime YOLO Monster Detection With Non Maximum Supression

Realtime-YOLO-Monster-Detection-With-Non-Maximum-Supression Table of Contents In

null 5 Oct 7, 2022
Facial Expression Detection In The Realtime
Facial Expression Detection In The Realtime

The human's facial expressions is very important to detect thier emotions and sentiment. It can be very efficient to use to make our computers make interviews. Furthermore, we have robots now can detect the human's emotions and based on thats take an action .etc. So, It will be better to provide a tool or model for this.

Adel El-Nabarawy 4 Mar 1, 2022
Drone detection using YOLOv5
Drone detection using YOLOv5

This drone detection system uses YOLOv5 which is a family of object detection architectures and we have trained the model on Drone Dataset. Overview I

Tushar Sarkar 27 Dec 20, 2022
YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset
YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset

YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset, and represents Ultralytics open-source research int

阿才 73 Dec 16, 2022
YOLOv5 detection interface - PyQt5 implementation
YOLOv5 detection interface - PyQt5 implementation

所有代码已上传，直接clone后，运行yolo_win.py即可开启界面。 2021/9/29：加入置信度选择 界面是在ultralytics的yolov5基础上建立的，界面使用pyqt5实现，内容较简单，娱乐而已。 功能： 模型选择 本地文件选择(视频图片均可) 开关摄像头

null 487 Dec 27, 2022
A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano
A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

yolov5-helmet-detection-python A Python implementation of Yolov5 to detect head or helmet in the wild in Jetson Xavier nx and Jetson nano. In Jetson X

null 12 Dec 5, 2022
YOLOv5 + ROS2 object detection package

YOLOv5-ROS YOLOv5 + ROS2 object detection package This program changes the input of detect.py (ultralytics/yolov5) to sensor_msgs/Image of ROS2. Requi

Ar-Ray 23 Dec 19, 2022
Comments
  • the issue on the windows

    the issue on the windows

    it doesn't work properly on windows, just like this image and for many peoples in a classroom, it doesn't work. just like this image so I want to the site of its application scenarios

    opened by xiaomengxin123 4
  • Detection on custom yolo weights

    Detection on custom yolo weights

    I have a custom trained yolo weights and there the person class is not having 0 class index. How to handle that? As far as I understood you are taking the 0th index and passing it to slowfast_r50, since person class is in 0th index at coco dataset

    opened by soumyadbanik 0
  • Real time testing

    Real time testing

    Hai @wufan-tb

    How to test with real-time videos from video surveillance. is that possible and how to test the modal with multiple video surveillance cameras

    Thanks

    opened by Chuttyboy 1
  • some Exception

    some Exception

    Exception: CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling cublasCreate(handle). Cache may be out of date, try force_reload=True or see https://github.com/ultralytics/yolov5/issues/36 for help.

    opened by a957815154 1
Owner
WuFan
WuFan
GitHub
TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios

TPH-YOLOv5 This repo is the implementation of "TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured

cv516Buaa 439 Dec 22, 2022
Multi-task yolov5 with detection and segmentation based on yolov5

YOLOv5DS Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0) decoupled head anchor free segmentation head README中文 Ablation

null 150 Dec 30, 2022
PyTorchVideo is a deeplearning library with a focus on video understanding work

PyTorchVideo is a deeplearning library with a focus on video understanding work. PytorchVideo provides resusable, modular and efficient components needed to accelerate the video understanding research. PyTorchVideo is developed using PyTorch and supports different deeplearning video components like video models, video datasets, and video-specific transforms.

Facebook Research 2.7k Jan 7, 2023
🍅🍅🍅YOLOv5-Lite: lighter, faster and easier to deploy. Evolved from yolov5 and the size of model is only 1.7M (int8) and 3.3M (fp16). It can reach 10+ FPS on the Raspberry Pi 4B when the input size is 320×320~

YOLOv5-Lite：lighter, faster and easier to deploy Perform a series of ablation experiments on yolov5 to make it lighter (smaller Flops, lower memory, a

pogg 1.5k Jan 5, 2023
Yolov5-lite - Minimal PyTorch implementation of YOLOv5

Yolov5-Lite: Minimal YOLOv5 + Deep Sort Overview This repo is a shortened versio

Kadir Nar 57 Nov 28, 2022
The official TensorFlow implementation of the paper Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition

Action Transformer A Self-Attention Model for Short-Time Human Action Recognition This repository contains the official TensorFlow implementation of t

PIC4SeRCentre 20 Jan 3, 2023
Allows including an action inside another action (by preprocessing the Yaml file). This is how composite actions should have worked.

actions-includes Allows including an action inside another action (by preprocessing the Yaml file). Instead of using uses or run in your action step,

Tim Ansell 70 Nov 4, 2022
Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21).

ACTION-Net Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21). Getting Started EgoGesture data folder struct

V-Sense 171 Dec 26, 2022
Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization' (ICCV-21 Oral)

Learning-Action-Completeness-from-Points Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal A

Pilhyeon Lee 67 Jan 3, 2023
Human Action Controller - A human action controller running on different platforms.

Human Action Controller (HAC) Goal A human action controller running on different platforms. Fun Easy-to-use Accurate Anywhere Fun Examples Mouse Cont

null 27 Jul 20, 2022