This repository contains the code for "SBEVNet: End-to-End Deep Stereo Layout Estimation" paper by Divam Gupta, Wei Pu, Trenton Tabor, Jeff Schneider

Overview

SBEVNet: End-to-End Deep Stereo Layout Estimation

This repository contains the code for "SBEVNet: End-to-End Deep Stereo Layout Estimation" paper by Divam Gupta, Wei Pu, Trenton Tabor, Jeff Schneider

Usage

Dependencies

pip install --upgrade git+https://github.com/divamgupta/pytorch-propane
pip install torch==1.4.0 torchvision==0.5.0
pip install opencv-python
pip install torchgeometry

Dataset and Directories

For the example we use the following directories:

  • Datasets : ./datasets/carla/ and ./datasets/kitti/
  • Weights : ./sbevnet_weights/carla and ./sbevnet_weights/kitti
  • Predictions : ./predictions/kitti ./predictions/carla

Download and unzip the datasets and place them in ./datasets directory

Training

cd <cloned_repo_path>

Training the model on the CARLA dataset:

pytorch_propane sbevnet train    \
 --model_name sbevnet_model --network_name sbevnet --dataset_name  sbevnet_dataset_main --dataset_split train \
 --eval_dataset_name "sbevnet_dataset_main" --eval_dataset_split test \
 --batch_size 3  --eval_batch_size 1 \
 --n_epochs 20   --overwrite_epochs true  \
 --datapath "datasets/carla/dataset.json" \
 --save_path "sbevnet_weights/carla/carla_save_0" \
 --image_w 512 \
 --image_h 288 \
 --max_disp 64 \
 --n_hmap 100 \
 --xmin 1 \
 --xmax 39 \
 --ymin -19 \
 --ymax 19 \
 --cx 256 \
 --cy 144 \
 --f 179.2531 \
 --tx 0.2 \
 --camera_ext_x 0.9 \
 --camera_ext_y -0.1 \
 --fixed_cam_confs true \
 --do_ipm_rgb true \
 --do_ipm_feats true  \
 --do_mask true --check_degenerate true 

Training the model on the KITTI dataset:

pytorch_propane sbevnet train    \
 --model_name sbevnet_model --network_name sbevnet --dataset_name  sbevnet_dataset_main --dataset_split train \
 --eval_dataset_name "sbevnet_dataset_main" --eval_dataset_split test \
 --batch_size 3  --eval_batch_size 1 \
 --n_epochs 40   --overwrite_epochs true  \
 --datapath "datasets/kitti/dataset.json" \
 --save_path "sbevnet_weights/kitti/kitti_save_0" \
 --image_w 640 \
 --image_h 256 \
 --max_disp 64 \
 --n_hmap 128 \
 --xmin 5.72 \
 --xmax 43.73 \
 --ymin -19 \
 --ymax 19 \
 --camera_ext_x 0 \
 --camera_ext_y 0 \
 --fixed_cam_confs false \
 --do_ipm_rgb true \
 --do_ipm_feats true  \
 --do_mask true --check_degenerate true 

Evaluation

Evaluating the model on the CARLA dataset:

pytorch_propane sbevnet eval_iou    \
 --model_name sbevnet_model --network_name sbevnet \
 --eval_dataset_name "sbevnet_dataset_main" --eval_dataset_split test --dataset_type carla \
 --eval_batch_size 1 \
 --datapath "datasets/carla/dataset.json" \
 --load_checkpoint_path "sbevnet_weights/carla/carla_save_0" \
 --image_w 512 \
 --image_h 288 \
 --max_disp 64 \
 --n_hmap 100 \
 --xmin 1 \
 --xmax 39 \
 --ymin -19 \
 --ymax 19 \
 --cx 256 \
 --cy 144 \
 --f 179.2531 \
 --tx 0.2 \
 --camera_ext_x 0.9 \
 --camera_ext_y -0.1 \
 --fixed_cam_confs true \
 --do_ipm_rgb true \
 --do_ipm_feats true  \
 --do_mask true 

Evaluating the model on the KITTI dataset:

pytorch_propane sbevnet eval_iou    \
 --model_name sbevnet_model --network_name sbevnet  \
 --eval_dataset_name "sbevnet_dataset_main" --eval_dataset_split test --dataset_type kitti \
 --eval_batch_size 1 \
 --datapath "datasets/kitti/dataset.json" \
 --load_checkpoint_path "sbevnet_weights/kitti/kitti_save_0" \
 --image_w 640 \
 --image_h 256 \
 --max_disp 64 \
 --n_hmap 128 \
 --xmin 5.72 \
 --xmax 43.73 \
 --ymin -19 \
 --ymax 19 \
 --camera_ext_x 0 \
 --camera_ext_y 0 \
 --fixed_cam_confs false \
 --do_ipm_rgb true \
 --do_ipm_feats true  \
 --do_mask true 

Save Predictions

Save predictions of the model on the CARLA dataset:

pytorch_propane sbevnet save_preds    \
 --model_name sbevnet_model --network_name sbevnet \
 --eval_dataset_name "sbevnet_dataset_main" --eval_dataset_split test --output_dir "predictions/kitti" \
 --eval_batch_size 1 \
 --datapath "datasets/carla/dataset.json" \
 --load_checkpoint_path "sbevnet_weights/carla/carla_save_0" \
 --image_w 512 \
 --image_h 288 \
 --max_disp 64 \
 --n_hmap 100 \
 --xmin 1 \
 --xmax 39 \
 --ymin -19 \
 --ymax 19 \
 --cx 256 \
 --cy 144 \
 --f 179.2531 \
 --tx 0.2 \
 --camera_ext_x 0.9 \
 --camera_ext_y -0.1 \
 --fixed_cam_confs true \
 --do_ipm_rgb true \
 --do_ipm_feats true  \
 --do_mask true 

Save predictions of the model on the KITTI dataset:

pytorch_propane sbevnet save_preds    \
 --model_name sbevnet_model --network_name sbevnet  \
 --eval_dataset_name "sbevnet_dataset_main" --eval_dataset_split test --output_dir "predictions/kitti" \
 --eval_batch_size 1 \
 --datapath "datasets/kitti/dataset.json" \
 --load_checkpoint_path "sbevnet_weights/kitti/kitti_save_0" \
 --image_w 640 \
 --image_h 256 \
 --max_disp 64 \
 --n_hmap 128 \
 --xmin 5.72 \
 --xmax 43.73 \
 --ymin -19 \
 --ymax 19 \
 --camera_ext_x 0 \
 --camera_ext_y 0 \
 --fixed_cam_confs false \
 --do_ipm_rgb true \
 --do_ipm_feats true  \
 --do_mask true 
You might also like...
This GitHub repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.'

About Repository This repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.' About Code

This repository contains the code for the paper
This repository contains the code for the paper "PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization"

PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization News: [2020/05/04] Added EGL rendering option for training data g

This repository contains the code and models necessary to replicate the results of paper:  How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective
This repository contains the code and models necessary to replicate the results of paper: How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective

Black-Box-Defense This repository contains the code and models necessary to replicate the results of our recent paper: How to Robustify Black-Box ML M

This repository contains the code and models necessary to replicate the results of paper:  How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective
This repository contains the code and models necessary to replicate the results of paper: How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective

Black-Box-Defense This repository contains the code and models necessary to replicate the results of our recent paper: How to Robustify Black-Box ML M

This repository contains the data and code for the paper
This repository contains the data and code for the paper "Diverse Text Generation via Variational Encoder-Decoder Models with Gaussian Process Priors" (SPNLP@ACL2022)

GP-VAE This repository provides datasets and code for preprocessing, training and testing models for the paper: Diverse Text Generation via Variationa

An image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testingAn image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testing
An image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testingAn image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testing

SVM Données Une base d’images contient 490 images pour l’apprentissage (400 voitures et 90 bateaux), et encore 21 images pour fait des tests. Prétrait

This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation

SO-Pose This repository contains codes of ICCV2021 paper: SO-Pose: Exploiting Self-Occlusion for Direct 6D Pose Estimation This paper is basically an

This repository contains the PyTorch implementation of the paper STaCK: Sentence Ordering with Temporal Commonsense Knowledge appearing at EMNLP 2021.
This repository contains the PyTorch implementation of the paper STaCK: Sentence Ordering with Temporal Commonsense Knowledge appearing at EMNLP 2021.

STaCK: Sentence Ordering with Temporal Commonsense Knowledge This repository contains the pytorch implementation of the paper STaCK: Sentence Ordering

This repository contains the implementation of the paper:
This repository contains the implementation of the paper: "Towards Frequency-Based Explanation for Robust CNN"

RobustFreqCNN About This repository contains the implementation of the paper "Towards Frequency-Based Explanation for Robust CNN" arxiv. It primarly d

Owner
Divam Gupta
Graduate student at Carnegie Mellon University | Former Research Fellow at Microsoft Research
Divam Gupta
Code for "MetaMorph: Learning Universal Controllers with Transformers", Gupta et al, ICLR 2022

MetaMorph: Learning Universal Controllers with Transformers This is the code for the paper MetaMorph: Learning Universal Controllers with Transformers

Agrim Gupta 50 Jan 3, 2023
This repository contains the code for the paper "Hierarchical Motion Understanding via Motion Programs"

Hierarchical Motion Understanding via Motion Programs (CVPR 2021) This repository contains the official implementation of: Hierarchical Motion Underst

Sumith Kulal 40 Dec 5, 2022
null 190 Jan 3, 2023
This repository contains the source code and data for reproducing results of Deep Continuous Clustering paper

Deep Continuous Clustering Introduction This is a Pytorch implementation of the DCC algorithms presented in the following paper (paper): Sohil Atul Sh

Sohil Shah 197 Nov 29, 2022
This repository contains a re-implementation of the code for the CVPR 2021 paper "Omnimatte: Associating Objects and Their Effects in Video."

Omnimatte in PyTorch This repository contains a re-implementation of the code for the CVPR 2021 paper "Omnimatte: Associating Objects and Their Effect

Erika Lu 728 Dec 28, 2022
This repository contains the source code for the paper "DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks",

DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks Project Page | Video | Presentation | Paper | Data L

Facebook Research 281 Dec 22, 2022
This repository contains the code and models for the following paper.

DC-ShadowNet Introduction This is an implementation of the following paper DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised

AuAgCu 65 Dec 27, 2022
This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis, accepted at EMNLP 2021.

MultiModal-InfoMax This repository contains the official implementation code of the paper Improving Multimodal Fusion with Hierarchical Mutual Informa

Deep Cognition and Language Research (DeCLaRe) Lab 89 Dec 26, 2022
This repository contains the code for the CVPR 2021 paper "GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields"

GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields Project Page | Paper | Supplementary | Video | Slides | Blog | Talk If

null 1.1k Dec 30, 2022
This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"

Differentiable Volumetric Rendering Paper | Supplementary | Spotlight Video | Blog Entry | Presentation | Interactive Slides | Project Page This repos

null 697 Jan 6, 2023