Libtorch yolov3 deepsort

Overview

Overview

It is for my undergrad thesis in Tsinghua University.

There are four modules in the project:

  • Detection: YOLOv3
  • Tracking: SORT and DeepSORT
  • Processing: Run detection and tracking, then display and save the results (a compressed video, a few snapshots for each target)
  • GUI: Display the results

YOLOv3

A Libtorch implementation of the YOLO v3 object detection algorithm, written with modern C++.

The code is based on the walktree.

The config file in .\models can be found at Darknet.

SORT

I also merged SORT to do tracking.

A similar software in Python is here, which also rewrite form the most starred version and SORT

DeepSORT

Recently I reimplement DeepSORT which employs another CNN for re-id. It seems it gives better result but also slows the program a bit. Also, a PyTorch version is available at ZQPei, thanks!

Performance

Currently on a GTX 1060 6G it consumes about 1G RAM and have 37 FPS.

The video I test is TownCentreXVID.avi.

GUI

With wxWidgets, I developed the GUI module for visualization of results.

Previously I used Dear ImGui. However, I do not think it suits my purpose.

Pre-trained network

This project uses pre-trained network weights from others

How to build

This project requires LibTorch, OpenCV, wxWidgets and CMake to build.

LibTorch can be easily integrated with CMake, but there are a lot of strange things...

On Ubuntu 16.04, I use apt install to install the others. Everything is fine. On Windows 10 + Visual Studio 2017, I use the latest stable version of the others from their official websites.

Snapshots

Here are some intermediate output from detection and tracking module: Detection Tracking

Here is the snapshot of processing module: Processing

Here is the snapshot of GUI module: GUI

Comments
  • cmake faied

    cmake faied

    it seems you reference the lib of libtorch and that makes this error ,but i am not sure how to fix this issue ,can you give me some help about this and i wil appreciate it. CMake Error at /home/suning/libtorch/share/cmake/Caffe2/public/utils.cmake:16 (add_library): add_library cannot create target "torch_library" because another target with the same name already exists. The existing target is an interface library created in source directory "/home/suning/libtorch-yolov3-deepsort-master/detection". See documentation for policy CMP0002 for more details.

    opened by Vincent630 4
  • lib torch安装

    lib torch安装

    请问:你的libtorch是如何安装的,我直接使用编译好的库,会报错“add_library cannot create target "caffe2_library" because another target with the same name already exists”

    opened by lcaikk1314 2
  • how to build on windows 10

    how to build on windows 10

    i downloaded all the and libs in Readme.txt , but i do not know how to build the project using cmake , what command should i write in command line ??? P.S. i am new to cmake and all these stuff

    @weixu000

    opened by ahmadalzoubi13579 1
  • ckpt.bin

    ckpt.bin

    Is weights/ckpt.bin in Extractor.cpp same ckpt.t7 from https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6 ?

    When I rename to .bin and copy it in the weights directory I get a segmentation fault.

    I am using Ubuntu 18.04 x86-64

    Thanks so much

    opened by danieljpb 1
  • Transfer torch::Tensor from cuda to cpu slow

    Transfer torch::Tensor from cuda to cpu slow

    When I test your repo i found that in line 72 of nn_matching.h have a trouble. when you call nn_cosine_distance if for loop (line 55), when i==0, time taken by .cpu() was >22000 micro second, but with other index, it took only 10-20 microseconds. If there is the way decrease that one, performance will increase dramatically.

    opened by datlt4 0
  • Installed  libtorch-cuda10-1.2.0,opencv4.0.1 . When I make an error, have anyone encountered and solved this problem?

    Installed libtorch-cuda10-1.2.0,opencv4.0.1 . When I make an error, have anyone encountered and solved this problem?

    In file included from /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:5:0: /home/libtorch-yolov3-deepsort/detection/src/letterbox.h: In function ‘std::array<long int, 2ul> letterbox_dim(c10::IntArrayRef, c10::IntArrayRef)’: /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:11:22: error: missing template arguments before ‘{’ token return std::array{int64_t(img_h * s), int64_t(img_w * s)}; ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:11:22: error: expected ‘;’ before ‘{’ token /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:11:61: error: expected ‘;’ before ‘}’ token return std::array{int64_t(img_h * s), int64_t(img_w * s)}; ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h: In function ‘cv::Mat letterbox_img(const cv::Mat&, c10::IntArrayRef)’: /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:16:9: error: expected unqualified-id before ‘[’ token auto[new_h, new_w] = letterbox_dim({img.rows, img.cols}, box); ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:16:60: error: expected primary-expression before ‘,’ token auto[new_h, new_w] = letterbox_dim({img.rows, img.cols}, box); ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:21:30: error: ‘new_h’ was not declared in this scope out({int((h - new_h) / 2), int((h - new_h) / 2 + new_h)}, ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:22:30: error: ‘new_w’ was not declared in this scope {int((w - new_w) / 2), int((w - new_w) / 2 + new_w)}), ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:22:72: error: no match for call to ‘(cv::Mat) (, )’ {int((w - new_w) / 2), int((w - new_w) / 2 + new_w)}), ^ In file included from /usr/local/include/opencv2/core/mat.hpp:3699:0, from /usr/local/include/opencv2/core.hpp:59, from /usr/local/include/opencv2/opencv.hpp:52, from /home/libtorch-yolov3-deepsort/detection/include/Detector.h:6, from /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:3: /usr/local/include/opencv2/core/mat.inl.hpp:904:5: note: candidate: cv::Mat cv::Mat::operator()(cv::Range, cv::Range) const Mat Mat::operator()( Range rowRange, Range colRange ) const /usr/local/include/opencv2/core/mat.inl.hpp:904:5: note: no known conversion for argument 1 from ‘’ to ‘cv::Range’ /usr/local/include/opencv2/core/mat.inl.hpp:910:5: note: candidate: cv::Mat cv::Mat::operator()(const Rect&) const Mat Mat::operator()( const Rect& roi ) const ^ /usr/local/include/opencv2/core/mat.inl.hpp:910:5: note: candidate expects 1 argument, 2 provided /usr/local/include/opencv2/core/mat.inl.hpp:916:5: note: candidate: cv::Mat cv::Mat::operator()(const cv::Range*) const Mat Mat::operator()(const Range* ranges) const ^ /usr/local/include/opencv2/core/mat.inl.hpp:916:5: note: candidate expects 1 argument, 2 provided /usr/local/include/opencv2/core/mat.inl.hpp:922:5: note: candidate: cv::Mat cv::Mat::operator()(const std::vectorcv::Range&) const Mat Mat::operator()(const std::vector& ranges) const ^ /usr/local/include/opencv2/core/mat.inl.hpp:922:5: note: candidate expects 1 argument, 2 provided In file included from /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:5:0: /home/libtorch-yolov3-deepsort/detection/src/letterbox.h: In function ‘void inv_letterbox_bbox(at::Tensor, c10::IntArrayRef, c10::IntArrayRef)’: /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:31:9: error: expected unqualified-id before ‘[’ token auto[new_h, new_w] = letterbox_dim(img_dim, box_dim); ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:33:34: error: ‘new_w’ was not declared in this scope bbox.select(1, 0).add(-(w - new_w) / 2).mul(1.0f * img_w / new_w); ^ /home/libtorch-yolov3-deepsort/detection/src/letterbox.h:36:34: error: ‘new_h’ was not declared in this scope bbox.select(1, 1).add_(-(h - new_h) / 2).mul_(1.0f * img_h / new_h); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp: In function ‘auto {anonymous}::threshold_confidence(at::Tensor, float)’: /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:14:13: error: expected unqualified-id before ‘[’ token auto[max_cls_score, max_cls] = pred.slice(1, 5).max(1); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:15:9: error: ‘max_cls_score’ was not declared in this scope max_cls_score *= pred.select(1, 4); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:22:32: error: ‘max_cls’ was not declared in this scope max_cls.index_select(0, index), ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp: In member function ‘std::vector<cv::Rect_ > Detector::detect(cv::Mat)’: /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:87:9: error: expected unqualified-id before ‘[’ token auto[bbox, cls, scr] = threshold_confidence(prediction, confidence_threshold); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:88:5: error: ‘bbox’ was not declared in this scope bbox = bbox.cpu(); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:89:5: error: ‘cls’ was not declared in this scope cls = cls.cpu(); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:90:5: error: ‘scr’ was not declared in this scope scr = scr.cpu(); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:99:35: error: expected primary-expression before ‘float’ auto bbox_acc = bbox.accessor<float, 2>(); ^ /home/libtorch-yolov3-deepsort/detection/src/Detector.cpp:100:33: error: expected primary-expression before ‘float’ auto scr_acc = scr.accessor<float, 1>(); ^ detection/CMakeFiles/detection.dir/build.make:89: recipe for target 'detection/CMakeFiles/detection.dir/src/Detector.cpp.o' failed make[2]: *** [detection/CMakeFiles/detection.dir/src/Detector.cpp.o] Error 1 CMakeFiles/Makefile2:151: recipe for target 'detection/CMakeFiles/detection.dir/all' failed make[1]: *** [detection/CMakeFiles/detection.dir/all] Error 2 Makefile:90: recipe for target 'all' failed make: *** [all] Error 2

    opened by yzqxy 0
  • Error when building with

    Error when building with "make" on Ubuntu 18.04

    Hello all,

    I need a bit of help with building the repository. My specifications are as follows:

    Ubuntu 18.04 libtorch v.1.0.0 OpenCV 3.2

    I cloned the repo, created a build folder and used cmake .. , which went well. When I use make, it gives me the following error: cmake .. -- Configuring done -- Generating done -- Build files have been written to: /home/andik/Schreibtisch/ML/libtorch-yolov3-deepsort/build andik@andik-VirtualBox:~/Schreibtisch/ML/libtorch-yolov3-deepsort/build$ make ..andik@andik-VirtualBox:~/Schreibtisch/ML/libtorch-yolov3-deepsort/build$ make Scanning dependencies of target detection [ 5%] Building CXX object detection/CMakeFiles/detection.dir/src/Darknet.cpp.o /home/andik/Schreibtisch/ML/libtorch-yolov3-deepsort/detection/src/Darknet.cpp:63:53: error: ‘torch::IntArrayRef’ has not been declared torch::Tensor forward(torch::Tensor prediction, torch::IntArrayRef inp_dim) { ^~~~~ /home/andik/Schreibtisch/ML/libtorch-yolov3-deepsort/detection/src/Darknet.cpp: In member function ‘at::Tensor DetectionLayerImpl::forward(at::Tensor, int)’: /home/andik/Schreibtisch/ML/libtorch-yolov3-deepsort/detection/src/Darknet.cpp:72:38: error: invalid types ‘int[int]’ for array subscript int64_t stride[] = {inp_dim[0] / grid_size[0], inp_dim[1] / grid_size[1]}; ^ /home/andik/Schreibtisch/ML/libtorch-yolov3-deepsort/detection/src/Darknet.cpp:72:65: error: invalid types ‘int[int]’ for array subscript int64_t stride[] = {inp_dim[0] / grid_size[0], inp_dim[1] / grid_size[1]}; ^ detection/CMakeFiles/detection.dir/build.make:62: recipe for target 'detection/CMakeFiles/detection.dir/src/Darknet.cpp.o' failed make[2]: *** [detection/CMakeFiles/detection.dir/src/Darknet.cpp.o] Error 1 CMakeFiles/Makefile2:85: recipe for target 'detection/CMakeFiles/detection.dir/all' failed make[1]: *** [detection/CMakeFiles/detection.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: *** [all] Error 2

    Has anyone experienced the same error? Best regards Andi

    opened by andikra 1
  • 如何调整跟踪参数?

    如何调整跟踪参数?

    首先感谢@weixu000做的很棒的工作! 我用yolov4+TensorRT+torch复现了这个项目,但我发现一个问题,在进行跟踪时,有时候tracker似乎并没有跟踪到目标。但是detector的检测是检测到的,并且置信度还挺高。而且看传给tracker的参数,也只是将rect传过去就可以,与置信度并无关系。 所以我想请教一下,应该如何调整跟踪参数,使得跟踪更加准确并且不会丢呢? 我看deepsort.cpp对iou_mat有一些阈值,是不是要调整这里? 麻烦您方便的时候给与解答 再次感谢@weixu000 !

    opened by opentld 1
Owner
Xu Wei
Xu Wei
Object tracking implemented with YOLOv4, DeepSort, and TensorFlow.

Object tracking implemented with YOLOv4, DeepSort, and TensorFlow. YOLOv4 is a state of the art algorithm that uses deep convolutional neural networks to perform object detections. We can take the output of YOLOv4 feed these object detections into Deep SORT (Simple Online and Realtime Tracking with a Deep Association Metric) in order to create a highly accurate object tracker.

The AI Guy 1.1k Dec 29, 2022
using yolox+deepsort for object-tracker

YOLOX_deepsort_tracker yolox+deepsort实现目标跟踪 最新的yolox尝尝鲜~~(yolox正处在频繁更新阶段,因此直接链接yolox仓库作为子模块) Install Clone the repository recursively: git clone --rec

null 245 Dec 26, 2022
Vehicles Counting using YOLOv4 + DeepSORT + Flask + Ngrok

A project for counting vehicles using YOLOv4 + DeepSORT + Flask + Ngrok

Duong Tran Thanh 37 Dec 16, 2022
StrongSORT: Make DeepSORT Great Again

StrongSORT StrongSORT: Make DeepSORT Great Again StrongSORT: Make DeepSORT Great Again Yunhao Du, Yang Song, Bo Yang, Yanyun Zhao arxiv 2202.13514 Abs

null 369 Jan 4, 2023
Minimal PyTorch implementation of YOLOv3

A minimal PyTorch implementation of YOLOv3, with support for training, inference and evaluation.

Erik Linder-Norén 6.9k Dec 29, 2022
A high-performance anchor-free YOLO. Exceeding yolov3~v5 with ONNX, TensorRT, NCNN, and Openvino supported.

YOLOX is an anchor-free version of YOLO, with a simpler design but better performance! It aims to bridge the gap between research and industrial communities. For more details, please refer to our report on Arxiv.

null 7.7k Jan 6, 2023
YOLOv3 in PyTorch > ONNX > CoreML > TFLite

This repository represents Ultralytics open-source research into future object detection methods, and incorporates lessons learned and best practices

Ultralytics 9.3k Jan 7, 2023
YOLOX is a high-performance anchor-free YOLO, exceeding yolov3~v5 with ONNX, TensorRT, ncnn, and OpenVINO supported.

Introduction YOLOX is an anchor-free version of YOLO, with a simpler design but better performance! It aims to bridge the gap between research and ind

null 7.7k Jan 3, 2023
Train a state-of-the-art yolov3 object detector from scratch!

TrainYourOwnYOLO: Building a Custom Object Detector from Scratch This repo let's you train a custom image detector using the state-of-the-art YOLOv3 c

AntonMu 616 Jan 8, 2023
Yolov3 pytorch implementation

YOLOV3 Pytorch实现 在bubbliiing大佬代码的基础上进行了修改,添加了部分注释。 预训练模型 预训练模型来源于bubbliiing。 链接:https://pan.baidu.com/s/1ncREw6Na9ycZptdxiVMApw 提取码:appk 训练自己的数据集 按照VO

null 4 Aug 27, 2022
Dataset para entrenamiento de yoloV3 para 4 clases

Deteccion de objetos en video Este repo basado en el proyecto PyTorch YOLOv3 para correr detección de objetos sobre video. Construí sobre este proyect

null 1 Nov 1, 2021
This project deals with the detection of skin lesions within the ISICs dataset using YOLOv3 Object Detection with Darknet.

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. Skin Lesion detection using YOLO This project deal

Lalith Veerabhadrappa Badiger 1 Nov 22, 2021
🔥 TensorFlow Code for technical report: "YOLOv3: An Incremental Improvement"

?? Are you looking for a new YOLOv3 implemented by TF2.0 ? If you hate the fucking tensorflow1.x very much, no worries! I have implemented a new YOLOv

null 3.6k Dec 26, 2022
Object Detection with YOLOv3

Object Detection with YOLOv3 Bu projede YOLOv3-608 modeli kullanılmıştır. Requirements Python 3.8 OpenCV Numpy Documentation Yolo ile ilgili detaylı b

Ayşe Konuş 0 Mar 27, 2022
Multiple custom object count and detection using YOLOv3-Tiny method

Electronic-Component-YOLOv3 Introduce This project created to detect, count, and recognize multiple custom object using YOLOv3-Tiny method. The target

Derwin Mahardika 2 Nov 14, 2022
Torchyolo - Yolov3 ve Yolov4 modellerin Pytorch uygulamasıdır

TORCHYOLO : Yolo Modellerin Pytorch Uygulaması Yapılacaklar: Yolov3 model.py ve

Kadir Nar 3 Aug 22, 2022
yolov5 deepsort 行人 车辆 跟踪 检测 计数

yolov5 deepsort 行人 车辆 跟踪 检测 计数 实现了 出/入 分别计数。 默认是 南/北 方向检测,若要检测不同位置和方向,可在 main.py 文件第13行和21行,修改2个polygon的点。 默认检测类别:行人、自行车、小汽车、摩托车、公交车、卡车。 检测类别可在 detect

null 554 Dec 30, 2022
Object tracking implemented with YOLOv4, DeepSort, and TensorFlow.

Object tracking implemented with YOLOv4, DeepSort, and TensorFlow. YOLOv4 is a state of the art algorithm that uses deep convolutional neural networks to perform object detections. We can take the output of YOLOv4 feed these object detections into Deep SORT (Simple Online and Realtime Tracking with a Deep Association Metric) in order to create a highly accurate object tracker.

The AI Guy 1.1k Dec 29, 2022
using yolox+deepsort for object-tracker

YOLOX_deepsort_tracker yolox+deepsort实现目标跟踪 最新的yolox尝尝鲜~~(yolox正处在频繁更新阶段,因此直接链接yolox仓库作为子模块) Install Clone the repository recursively: git clone --rec

null 245 Dec 26, 2022
Vehicles Counting using YOLOv4 + DeepSORT + Flask + Ngrok

A project for counting vehicles using YOLOv4 + DeepSORT + Flask + Ngrok

Duong Tran Thanh 37 Dec 16, 2022