SimpleDepthEstimation - An unified codebase for NN-based monocular depth estimation methods

Last update: Dec 13, 2022

Overview

SimpleDepthEstimation

Introduction

This is an unified codebase for NN-based monocular depth estimation methods, the framework is based on detectron2 (with a lot of modifications) and supports both supervised and self-supervised monocular depth estimation methods. The main goal for developing this repository is to help understand popular depth estimation papers, I tried my best to keep the code simple.

Environment:

clone this repo

SDE_ROOT=/path/to/SimpleDepthEstimation
git clone https://github.com/zzzxxxttt/SimpleDepthEstimation $SDE_ROOT
cd $SDE_ROOT

create a new conda environment and activate it

conda create -n sde python=3.6 
conda activate sde

install torch==1.8.0 and torchvision==0.9.0 follow the official instructions. (I haven't tried other pytorch versions)
install other requirements
```
pip install -r requirements.txt
```

Data preparation

KITTI:

Download and extract KITTI raw dataset, refined KITTI depth groundtruth, and eigen split files, then modify the data path in the config file.

Training

python path/to/project/train.py --num-gpus 2 --cfg path/to/config RUN_NAME run_name

Evaluation

python path/to/project/train.py --num-gpus 2 --cfg path/to/config --eval MODEL.WEIGHTS /path/to/checkpoint_file

Results:

KITTI:

model	type	config	abs rel err	sq rel err	rms	log rms	d1	d2	d3
ResNet-18	supervised	link	0.076	0.306	3.066	0.116	0.936	0.990	0.998
BTSNet (ResNet-50)	supervised	link	0.062	0.259	2.859	0.100	0.950	0.992	0.998
MonoDepth2 (ResNet-18)	self-supervised	link	0.118	0.735	4.517	0.163	0.860	0.974	0.994

Demo:

python tools/demo.py --cfg path/to/config --input path/to/image --output path/to/output_dir MODEL.WEIGHTS /path/to/checkpoint_file

Demo results:

Todo

add PackNet (I have added it, performance need verification)
add Dynamic Motion Learning (I have implemented it but still buggy, help welcome!)
support more datasets

Reference

You might also like...

Repository for "Toward Practical Monocular Indoor Depth Estimation" (CVPR 2022)

Toward Practical Monocular Indoor Depth Estimation Cho-Ying Wu, Jialiang Wang, Michael Hall, Ulrich Neumann, Shuochen Su [arXiv] [project site] DistDe

122 Dec 13, 2022

The implemention of Video Depth Estimation by Fusing Flow-to-Depth Proposals

Flow-to-depth (FDNet) video-depth-estimation This is the implementation of paper Video Depth Estimation by Fusing Flow-to-Depth Proposals Jiaxin Xie,

32 Jun 14, 2022

(CVPR 2022 - oral) Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry

Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry Official implementation of the paper Multi-View Depth Est

138 Dec 28, 2022

Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

Official PyTorch code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction. Guanglei Yang, Hao Tang, Mingli Ding, Nicu Sebe,

152 Dec 16, 2022

Categorical Depth Distribution Network for Monocular 3D Object Detection

CaDDN CaDDN is a monocular-based 3D object detection method. This repository is based off of [OpenPCDet]. Categorical Depth Distribution Network for M

289 Jan 5, 2023

AdelaiDepth is an open source toolbox for monocular depth prediction.

Adelaide Intelligent Machines (AIM) Group

743 Jan 1, 2023

Code and datasets for the paper "Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction" (RA-L, 2021)

Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction This is the code for the paper Combining E

69 Dec 26, 2022

Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR

Official implementation for paper "Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR"

72 Dec 9, 2022

Apply our monocular depth boosting to your own network!

MergeNet - Boost Your Own Depth Boost custom or edited monocular depth maps using MergeNet Input Original result After manual editing of base You can

142 Dec 17, 2022

Comments

How to add your depth network as a head in main detectron2?

Hi i also want to add box head and Sematic Seg Head, to make it to do multitask learning along with depth, how can i do that ? What all things i needed from your repo so that i can create a new network based which also perform depth estimation, semantic segmentation and depth estimatiion?

opened by Dheeru66k 1

SimpleDepthEstimation - An unified codebase for NN-based monocular depth estimation methods

Related tags

Overview

SimpleDepthEstimation

Introduction

Environment:

Data preparation

KITTI:

Training

Evaluation

Results:

KITTI:

Demo:

Todo

Reference

You might also like...

Repository for "Toward Practical Monocular Indoor Depth Estimation" (CVPR 2022)

The implemention of Video Depth Estimation by Fusing Flow-to-Depth Proposals

(CVPR 2022 - oral) Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry

Code for Transformers Solve Limited Receptive Field for Monocular Depth Prediction

Categorical Depth Distribution Network for Monocular 3D Object Detection

AdelaiDepth is an open source toolbox for monocular depth prediction.

Code and datasets for the paper "Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction" (RA-L, 2021)

Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR

Apply our monocular depth boosting to your own network!

Comments

How to add your depth network as a head in main detectron2?

Owner

Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals

Light-weight network, depth estimation, knowledge distillation, real-time depth estimation, auxiliary data.

Official implementation of the network presented in the paper "M4Depth: A motion-based approach for monocular depth estimation on video sequences"

Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging

[CVPR 2021] Monocular depth estimation using wavelets for efficiency

[ICCV 2021] Excavating the Potential Capacity of Self-Supervised Monocular Depth Estimation

This repo is for Self-Supervised Monocular Depth Estimation with Internal Feature Fusion(arXiv), BMVC2021

the official code for ICRA 2021 Paper: "Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation"

ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

ONNX-PackNet-SfM: Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX