Trained on Simulated Data, Tested in the Real World

livox

Last update: Nov 18, 2022

Related tags

Deep Learning livox_detection_simu

Overview

Livox Detection-simu V1.0: Trained on Simulated Data, Tested in the Real World [Livox Simu-dataset]

Introduction

Livox Detection-simu is a robust and real-time detection package trained on Livox Simu-dataset. It only uses 14k frames of simulated data for training, and performs effective detection in the real world. The inference time is about 50ms on 2080Ti for 200m*100m range detection.
We hope this project can help you make better use of Livox Simu-dataset. In order to improve the performance of the detector, data augmentation such as object insertion and background mix-up is necessary.

Demo

Dependencies

python3.6+
tensorflow1.13+ (tested on 1.13.0)
pybind11
ros

Installation

Clone this repository.
Clone pybind11 from pybind11.

$ cd utils/lib_cpp
$ git clone https://github.com/pybind/pybind11.git

Compile C++ module in utils/lib_cpp by running the following command.

$ mkdir build && cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ..
$ make

copy the lib_cpp.so to root directory:

$ cp lib_cpp.so ../../../

Download the pre_trained model and unzip it to the root directory.

Run

For sequence frame detection

Download the provided rosbags : rosbag and then

$ roscore

$ rviz -d ./config/show.rviz

$ python livox_detection_simu.py

$ rosbag play *.bag -r 0.1

The network inference time is around 25ms, but the point cloud data preprocessing module takes a lot of time based on python. If you want to get a faster real time detection demo, you can modify the point cloud data preprocessing module with c++.

To play with your own rosbag, please change your rosbag topic to /livox/lidar.

Support

You can get support from Livox with the following methods :

Send email to [email protected] with a clear description of your problem and your setup
Report issue on github

Comments

Gazebo plugin or the mathmatical formula of the pattern?

Hi, I am trying to apply Livox Horizon in autonomous exploraion task. However there is neither the gazebo plug-in for Livox horizon nor the principle of the scanning pattern of the horizon. Any ideas? So I may try to implement a gazebo plug-in myself.

opened by EpsAvlc 4
Tensorflow Error
My environment

ubuntu 18.04 LTS

python 3.6.9

tensorflow 1.15

I got the following error when I ran "python livox_detection_simu.py"

suzuki@ubuntu:~/Downloads/livox_detection_simu$ python3 livox_detection_simu.py WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see:

https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md

https://github.com/tensorflow/addons

https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue.

1008 224 30 WARNING:tensorflow:From /home/suzuki/Downloads/livox_detection_simu/networks/model.py:27: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

(1, 1008, 224, 30) WARNING:tensorflow:From /home/suzuki/venv/lib/python3.6/site-packages/tensorflow_core/contrib/layers/python/layers/layers.py:1057: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use layer.__call__ method instead. WARNING:tensorflow:From /home/suzuki/Downloads/livox_detection_simu/networks/model.py:70: The name tf.image.resize_bilinear is deprecated. Please use tf.compat.v1.image.resize_bilinear instead.

WARNING:tensorflow:From livox_detection_simu.py:66: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From livox_detection_simu.py:67: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From livox_detection_simu.py:71: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-04-28 01:42:50.203964: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2021-04-28 01:42:50.225177: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2592005000 Hz 2021-04-28 01:42:50.226874: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x4db39c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-04-28 01:42:50.226934: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-04-28 01:42:50.231211: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/suzuki/catkin_ws/devel/lib:/home/suzuki/Downloads/ws_livox/devel/lib:/opt/ros/melodic/lib 2021-04-28 01:42:50.231847: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303) 2021-04-28 01:42:50.231894: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (ubuntu): /proc/driver/nvidia/version does not exist Traceback (most recent call last): File "livox_detection_simu.py", line 320, in livox = Detector() File "livox_detection_simu.py", line 72, in init saver.restore(self.sess, cfg.MODEL_PATH) File "/home/suzuki/venv/lib/python3.6/site-packages/tensorflow_core/python/training/saver.py", line 1280, in restore if not checkpoint_management.checkpoint_exists_internal(checkpoint_prefix): File "/home/suzuki/venv/lib/python3.6/site-packages/tensorflow_core/python/training/checkpoint_management.py", line 366, in checkpoint_exists_internal if file_io.get_matching_files(pathname): File "/home/suzuki/venv/lib/python3.6/site-packages/tensorflow_core/python/lib/io/file_io.py", line 363, in get_matching_files return get_matching_files_v2(filename) File "/home/suzuki/venv/lib/python3.6/site-packages/tensorflow_core/python/lib/io/file_io.py", line 384, in get_matching_files_v2 compat.as_bytes(pattern)) tensorflow.python.framework.errors_impl.NotFoundError: model; No such file or directory

How do I get it to work?

Thank you for your answer
opened by suzuki3366 3
“python livox_detection_simu.py” error

Traceback (most recent call last): File "livox_detection_simu.py", line 7, in import lib_cpp ImportError: /home/xjllhy/Documents/livox_detection_simu/lib_cpp.so: undefined symbol: PyIndex_Check

opened by xjllhy 0
Is it a bug or typo in libcpp cal_result()
Hi ,

lib_cpp.cpp: https://github.com/Livox-SDK/livox_detection_simu/blob/master/utils/lib_cpp/lib_cpp.cpp#L398

if(is_obj>obj_th &&0)

It is weird "&& 0 ".

But in livox_detection repo, it is normal: if (is_obj > obj_th ).
opened by LHHCAL 0
tensorflow.python.framework.errors_impl.NotFoundError: model; No such file or directory

Hi guys, i get a trouble such as "tensorflow.python.framework.errors_impl.NotFoundError: model; No such file or directory" when I try to run "python3 livox_detection_simu.py" , Is there anybody met this question? could you tell me how to solve it? THANKS!

opened by applededipan 1
Can't detect pedestrian

Hi Livox guys, I am using livox detection sim to detect vehicle and pedestrian. But I found that using the pre-trained module you provided, it can not detect pedestrian. I also tried the other project "livox_detection" and its pre-trained model with the same bag file I used. It can detect pedestrian.But for vehicle detection, it is worse than livox_detection_sim. So I want to ask why?

opened by blueskych 2
How do I pre-train myself?

Livox - SDK, hello! How do you reuse livox_simu for pre-training?I want to test the AP and APH of the detection , so I want to repretrain, get a pretrained model. Looking forward to your reply!

opened by clytze0216 2

Owner

livox

https://www.livoxtech.com/

GitHub

Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

Real-ESRGAN Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data Ported from https://github.com/xinntao/Real-ESRGAN Depend

44 Dec 27, 2022

A complete end-to-end demonstration in which we collect training data in Unity and use that data to train a deep neural network to predict the pose of a cube. This model is then deployed in a simulated robotic pick-and-place task.

Object Pose Estimation Demo This tutorial will go through the steps necessary to perform pose estimation with a UR3 robotic arm in Unity. You’ll gain

187 Dec 24, 2022

A framework for analyzing computer vision models with simulated data

3DB: A framework for analyzing computer vision models with simulated data Paper Quickstart guide Blog post Installation Follow instructions on: https:

112 Jan 1, 2023

In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

Contrastive Learning of Object Representations Supervisor: Prof. Dr. Gemma Roig Institutions: Goethe University CVAI - Computational Vision & Artifici

6 Dec 8, 2022

This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

RGB2NIR_Experimental This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models

5 Jan 4, 2023

A toolkit for making real world machine learning and data analysis applications in C++

dlib C++ library Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real worl

11.6k Jan 1, 2023

A real world application of a Recurrent Neural Network on a binary classification of time series data

What is this This is a real world application of a Recurrent Neural Network on a binary classification of time series data. This project includes data

2 Jan 30, 2022

Annotate datasets with a semi-trained or fully trained YOLOv5 model

YOLOv5 Auto Annotator Annotate datasets with a semi-trained or fully trained YOLOv5 model Prerequisites Ubuntu >=20.04 Python >=3.7 System dependencie

3 May 14, 2022

A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution.

Awesome Pretrained StyleGAN2 A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution. Note the readme is a

1.1k Dec 24, 2022

Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing, Ant Colony Optimization Algorithm,Immune Algorithm, Artificial Fish Swarm Algorithm, Differential Evolution and TSP(Traveling salesman)

scikit-opt Swarm Intelligence in Python (Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing, Ant Colony Algorithm, Immune Algorithm,A

3.7k Jan 3, 2023

A 35mm camera, based on the Canonet G-III QL17 rangefinder, simulated in Python.

c is for Camera A 35mm camera, based on the Canonet G-III QL17 rangefinder, simulated in Python. The purpose of this project is to explore and underst

146 Sep 26, 2022

"Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback"

This is code repo for our EMNLP 2017 paper "Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback", which implements the A2C algorithm on top of a neural encoder-decoder model and benchmarks the combination under simulated noisy rewards.

131 Oct 21, 2022

Repo for FUZE project. I will also publish some Linux kernel LPE exploits for various real world kernel vulnerabilities here. the samples are uploaded for education purposes for red and blue teams.

Linux_kernel_exploits Some Linux kernel exploits for various real world kernel vulnerabilities here. More exploits are yet to come. This repo contains

472 Dec 21, 2022

Trained on Simulated Data, Tested in the Real World

Related tags

Overview

Livox Detection-simu V1.0: Trained on Simulated Data, Tested in the Real World [Livox Simu-dataset]

Introduction

Demo

Dependencies

Installation

Run

For sequence frame detection

Support

Comments

Gazebo plugin or the mathmatical formula of the pattern?

Tensorflow Error

“python livox_detection_simu.py” error

Is it a bug or typo in libcpp cal_result()

tensorflow.python.framework.errors_impl.NotFoundError: model; No such file or directory

Can't detect pedestrian

How do I pre-train myself?

Owner

livox

Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

A complete end-to-end demonstration in which we collect training data in Unity and use that data to train a deep neural network to predict the pose of a cube. This model is then deployed in a simulated robotic pick-and-place task.

A framework for analyzing computer vision models with simulated data

In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

A toolkit for making real world machine learning and data analysis applications in C++

A real world application of a Recurrent Neural Network on a binary classification of time series data

Annotate datasets with a semi-trained or fully trained YOLOv5 model

A collection of pre-trained StyleGAN2 models trained on different datasets at different resolution.

Genetic Algorithm, Particle Swarm Optimization, Simulated Annealing, Ant Colony Optimization Algorithm,Immune Algorithm, Artificial Fish Swarm Algorithm, Differential Evolution and TSP(Traveling salesman)

A 35mm camera, based on the Canonet G-III QL17 rangefinder, simulated in Python.

"Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback"

Simulated garment dataset for virtual try-on

PINN Burgers - 1D Burgers equation simulated by PINN

Real-world Anomaly Detection in Surveillance Videos- pytorch Re-implementation

The first dataset on shadow generation for the foreground object in real-world scenes.

HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset (ICCV 2021)

Learning Generative Models of Textured 3D Meshes from Real-World Images, ICCV 2021

Repo for FUZE project. I will also publish some Linux kernel LPE exploits for various real world kernel vulnerabilities here. the samples are uploaded for education purposes for red and blue teams.