A state of the art of new lightweight YOLO model implemented by TensorFlow 2.

Overview

CSL-YOLO: A New Lightweight Object Detection System for Edge Computing

license

This project provides a SOTA level lightweight YOLO called "Cross-Stage Lightweight YOLO"(CSL-YOLO),

it is achieving better detection performance with only 43% FLOPs and 52% parameters than Tiny-YOLOv4.

Paper Link: https://arxiv.org/abs/2107.04829

Requirements

How to Get Started?

#Predict
python3 main.py -p cfg/predict_coco.cfg

#Train
python3 main.py -t cfg/train_coco.cfg

#Eval
python3 main.py -ce cfg/eval_coco.cfg

WebCam DEMO(on CPU)

This DEMO runs on a pure CPU environment, the CPU is I7-6600U(2.6Ghz~3.4Ghz), the model scale is 224x224, and the FPS is about 10.

Please execute the following script to get this DEMO, the "camera_idx" in the cfg file represents the camera number you specified.

#Camera DEMO
python3 main.py -d cfg/demo_coco.cfg

More Info

Change Model Scale

The model's default scale is 224x224, if you want to change the scale to 320~512,

please go to cfg/XXXX.cfg and change the following two parts:

# input_shape=[512,512,3]
# out_hw_list=[[64,64],[48,48],[32,32],[24,24],[16,16]]
# input_shape=[416,416,3]
# out_hw_list=[[52,52],[39,39],[26,26],[20,20],[13,13]]
# input_shape=[320,320,3]
# out_hw_list=[[40,40],[30,30],[20,20],[15,15],[10,10]]
input_shape=[224,224,3]
out_hw_list=[[28,28],[21,21],[14,14],[10,10],[7,7]]

weight_path=weights/224_nolog.hdf5

                         |
                         | 224 to 320
                         V
                         
# input_shape=[512,512,3]
# out_hw_list=[[64,64],[48,48],[32,32],[24,24],[16,16]]
# input_shape=[416,416,3]
# out_hw_list=[[52,52],[39,39],[26,26],[20,20],[13,13]]
input_shape=[320,320,3]
out_hw_list=[[40,40],[30,30],[20,20],[15,15],[10,10]]
# input_shape=[224,224,3]
# out_hw_list=[[28,28],[21,21],[14,14],[10,10],[7,7]]

weight_path=weights/320_nolog.hdf5

Fully Dataset

The entire MS-COCO data set is too large, here only a few pictures are stored for DEMO,

if you need complete data, please download on this page.

Our Data Format

We did not use the official format of MS-COCO, we expressed a bounding box as following:

[ left_top_x<float>, left_top_y<float>, w<float>, h<float>, confidence<float>, class<str> ]

The bounding boxes contained in a picture are represented by single json file.

For detailed format, please refer to the json file in "data/coco/train/json".

AP Performance on MS-COCO

For detailed COCO report, please refer to "mscoco_result".

TODOs

  • Improve the calculator script of FLOPs.
  • Using Focal Loss will cause overfitting, we need to explore the reasons.
Comments
  • Jetson Nano:  Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB

    Jetson Nano: Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB

    I am running on Jetson Nano 2GB, Jetpack 4.6.1, tensorflow-2.7.0+nv22.1 Here's error message: [ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1 2022-04-17 14:04:31.889463: I tensorflow/stream_executor/cuda/cuda_dnn.cc:377] Loaded cuDNN version 8201 2022-04-17 14:04:33.712375: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.07MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:04:49.653386: W tensorflow/stream_executor/gpu/asm_compiler.cc:111] *** WARNING *** You are using ptxas 10.2.300, which is older than 11.1. ptxas before 11.1 is known to miscompile XLA code, leading to incorrect results or invalid-address errors.

    You may not need to update to CUDA 11.1; cherry-picking the ptxas binary is often sufficient. 2022-04-17 14:04:50.134582: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:04:50.184189: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.07MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:04:50.184357: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.08MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:05:19.685884: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:05:19.776200: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:05:19.821245: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:05:20.067985: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:05:20.114925: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:05:20.685095: W tensorflow/core/common_runtime/bfc_allocator.cc:275] Allocator (GPU_0_bfc) ran out of memory trying to allocate 16.00MiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available. 2022-04-17 14:05:26.275044: F tensorflow/core/kernels/image/resize_bilinear_op_gpu.cu.cc:445] Non-OK-status: GpuLaunchKernel(kernel, config.block_count, config.thread_per_block, 0, d.stream(), config.virtual_thread_count, images.data(), height_scale, width_scale, batch, in_height, in_width, channels, out_height, out_width, output.data()) status: INTERNAL: too many resources requested for launch Aborted (core dumped)

    opened by rkuo2000 0
  • 数据集格式请教

    数据集格式请教

    我当前有训练集和验证集,具体格式如下图所示,如何转成CSP-YOLO可训练的数据格式? image

    其中,txt格式如下所示 class_id cx/w cy/h box_w/w box_h/h 0 0.510814878287011 0.33199194073677063 0.04678068331669093 0.12072434276342392

    opened by lixiangMindSpore 0
  • 请教网络设计问题

    请教网络设计问题

    您好,因为自己是非科班,老师也没经验,自己研究进度缓慢。最近一直在优化yolov4-tiny,关于网络设计方面,想请教几个问题: (1)如何快速验证网络结构是否更优?我一般用VOC2007训练看结果,但也需要训练五六个小时,感觉试错成本太高 (2)网络在CPU和GPU上运行是有区别的,在CPU上更关注低Flpos和少分支结构,这个理解对吗? (3)因为网络backbone得到了大量简化,所以yolo-head要采用更多的scale,而不是原始yolov4-tiny的2层scale,这个理解对吗? (4)yolov4-tiny用最简单的结构实现了很好的效果,而我们人工设计的结构感觉都是在反复试错,您认为呢? Sincerely, Best wishes

    opened by forestguan 6
  • 關於精度和速度方面的問題

    關於精度和速度方面的問題

    作者你好,我在rtx3090上測試了一下fps,發現只能跑到二三十幀,yolo4tiny能達到一百五六十幀,是不是我設置方面有錯誤? 還有精度,我訓練了自己的數據集,用224224的初始模型,訓練了200個epoch,每個epoch是 batch=16,step=46,因爲訓練集就700多張圖片,但是最後map只有66%,我用tiny能達到94%,後來我用512512的跑了50個epoch準確率直接為0。。。標簽方面位置信息是雙精度的,這一點可能和你給的有差別,其他應該都差不多

    opened by wuminjie12 4
  • get error in loading datasets

    get error in loading datasets

    Traceback (most recent call last): File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/project/train/src_repo/CSL-YOLO/tools/thread_pool.py", line 22, in _Thread if(parameter==None):result=function() File "/project/train/src_repo/CSL-YOLO/data_generator.py", line 180, in _ReadFunction batch_data=self._data_gen() File "/project/train/src_repo/CSL-YOLO/data_generator.py", line 31, in call return self.Read() File "/project/train/src_repo/CSL-YOLO/data_generator.py", line 142, in Read img_1,bboxes_1=self._GetImgAndBboxes(file_name_1) File "/project/train/src_repo/CSL-YOLO/data_generator.py", line 46, in _GetImgAndBboxes bboxes=JSON2Bboxes(json_path) File "/project/train/src_repo/CSL-YOLO/tools/bboxes_io.py", line 4, in JSON2Bboxes with open(json_path,"r") as json_fin: FileNotFoundError: [Errno 2] No such file or directory: './dataset/coco/train/json/.json'

    By the way, the description about how to train is wrong

    opened by jiujiangluck 7
Owner
Miles Zhang
Miles Zhang
Yolo ros - YOLO-ROS for HUAWEI ATLAS200

YOLO-ROS YOLO-ROS for NVIDIA YOLO-ROS for HUAWEI ATLAS200, please checkout for b

ChrisLiu 5 Oct 18, 2022
Yolo object detection - Yolo object detection with python

How to run download required files make build_image make download Docker versio

null 3 Jan 26, 2022
LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.

LWCC: A LightWeight Crowd Counting library for Python LWCC is a lightweight crowd counting framework for Python. It wraps four state-of-the-art models

Matija Teršek 39 Dec 28, 2022
😇A pyTorch implementation of the DeepMoji model: state-of-the-art deep learning model for analyzing sentiment, emotion, sarcasm etc

------ Update September 2018 ------ It's been a year since TorchMoji and DeepMoji were released. We're trying to understand how it's being used such t

Hugging Face 865 Dec 24, 2022
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow ?? Transformers provides thousands of pretrained mo

Hugging Face 77.2k Jan 2, 2023
Implementation of the state-of-the-art vision transformers with tensorflow

ViT Tensorflow This repository contains the tensorflow implementation of the state-of-the-art vision transformers (a category of computer vision model

Mohammadmahdi NouriBorji 2 Mar 16, 2022
Based on Yolo's low-power, ultra-lightweight universal target detection algorithm, the parameter is only 250k, and the speed of the smart phone mobile terminal can reach ~300fps+

Based on Yolo's low-power, ultra-lightweight universal target detection algorithm, the parameter is only 250k, and the speed of the smart phone mobile terminal can reach ~300fps+

null 567 Dec 26, 2022
This project deploys a yolo fastest model in the form of tflite on raspberry 3b+. The model is from another repository of mine called -Trash-Classification-Car

Deploy-yolo-fastest-tflite-on-raspberry 觉得有用的话可以顺手点个star嗷 这个项目将垃圾分类小车中的tflite模型移植到了树莓派3b+上面。 该项目主要是为了记录在树莓派部署yolo fastest tflite的流程 (之后有时间会尝试用C++部署来提升

null 7 Aug 16, 2022
Implementation of the state of the art beat-detection, downbeat-detection and tempo-estimation model

The ISMIR 2020 Beat Detection, Downbeat Detection and Tempo Estimation Model Implementation. This is an implementation in TensorFlow to implement the

Koen van den Brink 1 Nov 12, 2021
In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

Contrastive Learning of Object Representations Supervisor: Prof. Dr. Gemma Roig Institutions: Goethe University CVAI - Computational Vision & Artifici

Dirk Neuhäuser 6 Dec 8, 2022
Lightweight mmm - Lightweight (Bayesian) Media Mix Model

Lightweight (Bayesian) Media Mix Model This is not an official Google product. L

Google 342 Jan 3, 2023
Object detection using yolo-tiny model and opencv used as backend

Object detection Algorithm used : Yolo algorithm Backend : opencv Library required: opencv = 4.5.4-dev' Quick Overview about structure 1) main.py Load

null 2 Jul 6, 2022
State of the Art Neural Networks for Deep Learning

pyradox This python library helps you with implementing various state of the art neural networks in a totally customizable fashion using Tensorflow 2

Ritvik Rastogi 60 May 29, 2022
Code for paper "A Critical Assessment of State-of-the-Art in Entity Alignment" (https://arxiv.org/abs/2010.16314)

A Critical Assessment of State-of-the-Art in Entity Alignment This repository contains the source code for the paper A Critical Assessment of State-of

Max Berrendorf 16 Oct 14, 2022
Quickly comparing your image classification models with the state-of-the-art models (such as DenseNet, ResNet, ...)

Image Classification Project Killer in PyTorch This repo is designed for those who want to start their experiments two days before the deadline and ki

null 349 Dec 8, 2022
State of the art Semantic Sentence Embeddings

Contrastive Tension State of the art Semantic Sentence Embeddings Published Paper · Huggingface Models · Report Bug Overview This is the official code

Fredrik Carlsson 88 Dec 30, 2022
QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

null 152 Jan 2, 2023
LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models

LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models. Developers can reproduce these SOTA methods and build their own methods.

TuZheng 405 Jan 4, 2023
tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series classification, regression and forecasting.

Time series Timeseries Deep Learning Pytorch fastai - State-of-the-art Deep Learning with Time Series and Sequences in Pytorch / fastai

timeseriesAI 2.8k Jan 8, 2023