Safe Local Motion Planning with Self-Supervised Freespace Forecasting, CVPR 2021

Related tags

Deep Learning ff
Overview

Teaser

Safe Local Motion Planning with Self-Supervised Freespace Forecasting

By Peiyun Hu, Aaron Huang, John Dolan, David Held, and Deva Ramanan

Citing us

You can find our paper on CVF Open Access. If you find our work useful, please consider citing:

@inproceedings{hu2021safe,
  title={Safe Local Motion Planning with Self-Supervised Freespace Forecasting},
  author={Hu, Peiyun and Huang, Aaron and Dolan, John and Held, David and Ramanan, Deva},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={12732--12741},
  year={2021}
}

Setup

  • Download nuScenes dataset, including the CANBus extension, as we will use the recorded vehicle state data for trajectory sampling. (Tip: the code assumes they are stored under /data/nuscenes.)
  • Install packages and libraries (via conda if possible), including torch, torchvision, tensorboard, cudatoolkit-11.1, pcl>=1.9, pybind11, eigen3, cmake>=3.10, scikit-image, nuscenes-devkit. (Tip: verify location of python binary with which python.)
  • Compile code for Lidar point cloud ground segmentation under lib/grndseg using CMake.

Preprocessing

  • Run preprocess.py to generate ground segmentations
  • Run precast.py to generate future visible freespace maps
  • Run rasterize.py to generate BEV object occupancy maps and object "shadow" maps.

Training

Refer to train.py.

Testing

Refer to test.py.

Acknowledgements

Thanks @tarashakhurana for help with README.

You might also like...
This is the official repo for TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations at CVPR'21. According to some product reasons, we are not planning to release the training/testing codes and models. However, we will release the dataset and the scripts to prepare the dataset.
This is the official repo for TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations at CVPR'21. According to some product reasons, we are not planning to release the training/testing codes and models. However, we will release the dataset and the scripts to prepare the dataset.

TransFill-Reference-Inpainting This is the official repo for TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transf

Yuqian Zhou 80 Dec 8, 2022
Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021.
Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021.

Dense Contrastive Learning for Self-Supervised Visual Pre-Training This project hosts the code for implementing the DenseCL algorithm for se

Xinlong Wang 491 Jan 3, 2023
Official code for the CVPR 2021 paper "How Well Do Self-Supervised Models Transfer?"

How Well Do Self-Supervised Models Transfer? This repository hosts the code for the experiments in the CVPR 2021 paper How Well Do Self-Supervised Mod

Linus Ericsson 157 Dec 16, 2022
Self-Supervised Multi-Frame Monocular Scene Flow (CVPR 2021)
Self-Supervised Multi-Frame Monocular Scene Flow (CVPR 2021)

Self-Supervised Multi-Frame Monocular Scene Flow 3D visualization of estimated depth and scene flow (overlayed with input image) from temporally conse

Visual Inference Lab @TU Darmstadt 85 Dec 22, 2022
Self-supervised Augmentation Consistency for Adapting Semantic Segmentation (CVPR 2021)
Self-supervised Augmentation Consistency for Adapting Semantic Segmentation (CVPR 2021)

Self-supervised Augmentation Consistency for Adapting Semantic Segmentation This repository contains the official implementation of our paper: Self-su

Visual Inference Lab @TU Darmstadt 132 Dec 21, 2022
Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

One Thing One Click One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR2021) Code for the paper One Thi

null 44 Dec 12, 2022
S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration (CVPR 2021)
S2-BNN: Bridging the Gap Between Self-Supervised Real and 1-bit Neural Networks via Guided Distribution Calibration (CVPR 2021)

S2-BNN (Self-supervised Binary Neural Networks Using Distillation Loss) This is the official pytorch implementation of our paper: "S2-BNN: Bridging th

Zhiqiang Shen 52 Dec 24, 2022
Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms
Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms

LESA Introduction This repository contains the official implementation of Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Cont

Chenglin Yang 20 Dec 31, 2021
Infrastructure as Code (IaC) for a self-hosted version of Gnosis Safe on AWS
Infrastructure as Code (IaC) for a self-hosted version of Gnosis Safe on AWS

Welcome to Yearn Gnosis Safe! Setting up your local environment Infrastructure Deploying Gnosis Safe Prerequisites 1. Create infrastructure for secret

Numan 16 Jul 18, 2022
Comments
  • Sample question

    Sample question

    Thanks for sharing your code! I see that your sampler.py code contains TODO statements. Does it mean that this code is not the final version? I also visualized the results of the sampler code, and I can see that there is still a large gap between the clothoid curve and the straight line that has not been sampled. How to solve this question? thanks~~

    Screenshot from 2021-09-29 12-03-29

    opened by charleshsc 0
  • Fail while running precast.py file

    Fail while running precast.py file

    Hi, I'm a newbie studying about using deep learning to predict future trajectory on cars. I have been studying your paper, and I am now starting to run your code. I made a dockerfile to run this code by using nuscenes' mini data, and I succeed on running "preprocess.py", but I failed while running "precast.py".

    I got this error, " keyerror : 'train on all sweeps' ", and I would like to get some advice on how to solve this error. Also if possible, could you explain the order in which code should be executed?

    It would be a great honor to run your code by your help.

    (base) root@b1d627403c9b:/usr/src/app# python -W ignore precast.py Using /root/.cache/torch_extensions as PyTorch extensions root... Detected CUDA files, patching ldflags Emitting ninja build file /root/.cache/torch_extensions/raycaster/build.ninja... Building extension module raycaster... Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N) [1/2] /usr/local/cuda/bin/nvcc -DTORCH_EXTENSION_NAME=raycaster -DTORCH_API_INCLUDE_EXTENSION_H -isystem /opt/conda/lib/python3.6/site-packages/torch/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/torch/csrc/api/include -isystem /opt/conda/lib/python3.6/site-packages/torch/include/TH -isystem /opt/conda/lib/python3.6/site-packages/torch/include/THC -isystem /usr/local/cuda/include -isystem /opt/conda/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_70,code=sm_70 -gencode=arch=compute_52,code=sm_52 -gencode=arch=compute_61,code=sm_61 -gencode=arch=compute_86,code=sm_86 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_75,code=sm_75 --compiler-options '-fPIC' -std=c++14 -c /usr/src/app/lib/raycast/raycaster.cu -o raycaster.cuda.o [2/2] c++ raycaster.o raycaster.cuda.o -shared -L/opt/conda/lib/python3.6/site-packages/torch/lib -lc10 -lc10_cuda -ltorch_cpu -ltorch_cuda -ltorch -ltorch_python -L/usr/local/cuda/lib64 -lcudart -o raycaster.so Loading extension module raycaster...

    Loading NuScenes tables for version v1.0-mini... Loading nuScenes-lidarseg... 32 category, 8 attribute, 4 visibility, 911 instance, 12 sensor, 120 calibrated_sensor, 31206 ego_pose, 8 log, 10 scene, 404 sample, 31206 sample_data, 18538 sample_annotation, 4 map, 404 lidarseg, Done loading in 0.375 seconds.

    Reverse indexing ... Done reverse indexing in 0.1 seconds.

    Traceback (most recent call last): File "precast.py", line 29, in dataset = nuScenesDataset(nusc, "train", dataset_kwargs) File "/usr/src/app/data.py", line 106, in init self.train_on_all_sweeps = kwargs["train_on_all_sweeps"] KeyError: 'train_on_all_sweeps'

    opened by diane416 0
  • Fail to compile code for Lidar point cloud ground segmentation

    Fail to compile code for Lidar point cloud ground segmentation

    Thanks for sharing the code! When I run preprocess.py, I have the following error:

    Traceback (most recent call last): File "preprocess.py", line 11, in from lib.grndseg import segmentation ImportError: cannot import name 'segmentation' from 'lib.grndseg' (unknown location)

    Does it mean I didn't compile the code in lib/grndseg successfully? Thanks!

    opened by Jimuyangz 1
  • ImportError: dynamic module does not define module export function (PyInit_segmentation)

    ImportError: dynamic module does not define module export function (PyInit_segmentation)

    Thanks for sharing your code! I encounter the problem about compile grndseg

    ImportError: dynamic module does not define module export function (PyInit_segmentation)

    How to solve it? Thanks

    opened by gzblue07 2
Owner
Peiyun Hu
Peiyun Hu
GitHub
(CVPR 2022) A minimalistic mapless end-to-end stack for joint perception, prediction, planning and control for self driving.

LAV Learning from All Vehicles Dian Chen, Philipp Krähenbühl CVPR 2022 (also arXiV 2203.11934) This repo contains code for paper Learning from all veh

Dian Chen 300 Dec 15, 2022
[CVPR 2021] "The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models" Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Michael Carbin, Zhangyang Wang

The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Models Codes for this paper The Lottery Tickets Hypo

VITA 59 Dec 28, 2022
Multivariate Time Series Forecasting with efficient Transformers. Code for the paper "Long-Range Transformers for Dynamic Spatiotemporal Forecasting."

Spacetimeformer Multivariate Forecasting This repository contains the code for the paper, "Long-Range Transformers for Dynamic Spatiotemporal Forecast

QData 440 Jan 2, 2023
Event-forecasting - Event Forecasting Algorithms With Python

event-forecasting Event Forecasting Algorithms Theory Correlating events in comp

Intellia ICT 4 Feb 15, 2022
Forecasting for knowable future events using Bayesian informative priors (forecasting with judgmental-adjustment).

What is judgyprophet? judgyprophet is a Bayesian forecasting algorithm based on Prophet, that enables forecasting while using information known by the

AstraZeneca 56 Oct 26, 2022
The Self-Supervised Learner can be used to train a classifier with fewer labeled examples needed using self-supervised learning.

Published by SpaceML • About SpaceML • Quick Colab Example Self-Supervised Learner The Self-Supervised Learner can be used to train a classifier with

SpaceML 92 Nov 30, 2022
An all-in-one application to visualize multiple different local path planning algorithms

Table of Contents Table of Contents Local Planner Visualization Project (LPVP) Features Installation/Usage Local Planners Probabilistic Roadmap (PRM)

Abdur Javaid 47 Dec 30, 2022
Implementation of Self-supervised Graph-level Representation Learning with Local and Global Structure (ICML 2021).

Self-supervised Graph-level Representation Learning with Local and Global Structure Introduction This project is an implementation of ``Self-supervise

MilaGraph 50 Dec 9, 2022
Video Autoencoder: self-supervised disentanglement of 3D structure and motion

Video Autoencoder: self-supervised disentanglement of 3D structure and motion This repository contains the code (in PyTorch) for the model introduced

null 157 Dec 22, 2022
[CVPR 2021] Forecasting the panoptic segmentation of future video frames

Panoptic Segmentation Forecasting Colin Graber, Grace Tsai, Michael Firman, Gabriel Brostow, Alexander Schwing - CVPR 2021 [Link to paper] We propose

Niantic Labs 44 Nov 29, 2022