Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX

Overview

ONNX-MobileStereoNet

Python scripts for performing stereo depth estimation using the MobileStereoNet model in ONNX

MobileStereoNet depth estimation ONNX Stereo depth estimation on the cones images from the Middlebury dataset (https://vision.middlebury.edu/stereo/data/scenes2003/)

Requirements

  • Check the requirements.txt file. Additionally, pafy and youtube-dl are required for youtube video inference.

Installation

pip install -r requirements.txt
pip install pafy youtube-dl

ONNX model

The original models were converted to different formats (including .onnx) by PINTO0309, the models can be found in his repository.

Original Pytorch model

The Pytorch pretrained model was taken from the original repository.

Examples

  • Image inference:
python image_depth_estimation.py 
  • Video inference:
python video_depth_estimation.py
  • DrivingStereo dataset inference:
python driving_sereo_test.py

Inference video Example

MobileStereoNet depth estimation ONNX

References:

You might also like...
Python scripts for performing lane detection using the LSTR model in ONNX
Python scripts for performing lane detection using the LSTR model in ONNX

ONNX LSTR Lane Detection Python scripts for performing lane detection using the Lane Shape Prediction with Transformers (LSTR) model in ONNX. Requirem

Python scripts for performing road segemtnation and car detection using the HybridNets multitask model in ONNX.
Python scripts for performing road segemtnation and car detection using the HybridNets multitask model in ONNX.

ONNX-HybridNets-Multitask-Road-Detection Python scripts for performing road segemtnation and car detection using the HybridNets multitask model in ONN

Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.
Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX.

ONNX Object Localization Network Python scripts performing class agnostic object localization using the Object Localization Network model in ONNX. Ori

Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models
Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute dept

Light-weight network, depth estimation, knowledge distillation, real-time depth estimation, auxiliary data.
Light-weight network, depth estimation, knowledge distillation, real-time depth estimation, auxiliary data.

light-weight-depth-estimation Boosting Light-Weight Depth Estimation Via Knowledge Distillation, https://arxiv.org/abs/2105.06143 Junjie Hu, Chenyou F

Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX.
Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX.

Python scripts for performing object detection with the 1000 labels of the ImageNet dataset in ONNX. The repository combines a class agnostic object localizer to first detect the objects in the image, and next a ResNet50 model trained on ImageNet is used to label each box.

Honours project, on creating a depth estimation map from two stereo images of featureless regions

image-processing This module generates depth maps for shape-blocked-out images Install If working with anaconda, then from the root directory: conda e

Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals
Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals

LapDepth-release This repository is a Pytorch implementation of the paper "Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals" M

The implemention of Video Depth Estimation by Fusing Flow-to-Depth Proposals

Flow-to-depth (FDNet) video-depth-estimation This is the implementation of paper Video Depth Estimation by Fusing Flow-to-Depth Proposals Jiaxin Xie,

Comments
  • Convert to ONNX model

    Convert to ONNX model

    Hi @ibaiGorordo ,

    Thank you for your great work !.

    I've tried to convert MobileStereoNet Pytorch model to ONNX model by using torch onnx export. However the ONNX model size is 10 times bigger than Pytorch model size. (i.e. 300Mbs, compare to 29Mbs).

    The script which I used to convert to onnx model:

    import torch
    import onnx
    from onnxsim import simplify
    from models import MSNet2D
    
    if __name__ == '__main__':
    
        # Load models
        maxdisp = 192
    
        model = MSNet2D(maxdisp)
        model.cuda()
    
        print('load model')
        model_path = 'weights/MSNet2D_SF_KITTI2015.ckpt'
        state_dict = torch.load(model_path)
    
        pretrained_dict = {key.replace("module.", ""): value for key, value in state_dict['model'].items()}
        model.load_state_dict(pretrained_dict)
        model.eval()
      
        W = 528
        H = 240
        onnx_file = f"mobileStereoNet_{H}x{W}.onnx"
        onnx_simp_file = f"mobileStereoNet_{H}x{W}_simp.onnx"
        
        x = torch.randn(1, 3, H, W).cuda()
        torch.onnx.export(model,
                                    args=(x, x),
                                    f=onnx_file,
                                    opset_version=12)
    
        print("save model {} successfully".format(onnx_file))
    
        model_load = onnx.load(onnx_file)
        model_simp, check = simplify(model_load)
        onnx.save(model_simp, onnx_simp_file)
    
        print("save model {} successfully".format(onnx_simp_file))
    
    

    Could you hint me how did you convert MobileStereoNet Pytorch to ONNX ?

    Many thanks in advance !.

    Regards, Michael

    opened by michaelnguyen11 3
  • Converting ONNX to TensorRT(.trt)

    Converting ONNX to TensorRT(.trt)

    Firstly, thank you for providing excellent work.

    I was about to work on converting the mobilestereonet onnx model to TensorRT model to improve inference speed. Even though I confirmed my own model was well converted to onnx using your work, but issues come up as I tried to convert the onnx to a TensorRT model.

    Have you tried the conversion? If so, it will be great if you could share your process since I'm struggling with various issues. I've tried TRT6 in Ubuntu18.04, CUDA10.1 environment.

    Thanks again!

    opened by ejhung 2
Owner
Ibai Gorordo
Passionate about sensors, technology and their potential to help people.
Ibai Gorordo
Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch .

PyTorch-High-Res-Stereo-Depth-Estimation Python scripts form performing stereo depth estimation using the high res stereo model in PyTorch. Stereo dep

Ibai Gorordo 26 Nov 24, 2022
Python scripts form performing stereo depth estimation using the CoEx model in ONNX.

ONNX-CoEx-Stereo-Depth-estimation Python scripts form performing stereo depth estimation using the CoEx model in ONNX. Stereo depth estimation on the

Ibai Gorordo 8 Dec 29, 2022
Python scripts form performing stereo depth estimation using the HITNET model in ONNX.

ONNX-HITNET-Stereo-Depth-estimation Python scripts form performing stereo depth estimation using the HITNET model in ONNX. Stereo depth estimation on

Ibai Gorordo 30 Nov 8, 2022
ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

Ibai Gorordo 18 Nov 6, 2022
ONNX-PackNet-SfM: Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Python scripts for performing monocular depth estimation using the PackNet-SfM model in ONNX

Ibai Gorordo 14 Dec 9, 2022
Python scripts for performing stereo depth estimation using the HITNET Tensorflow model.

HITNET-Stereo-Depth-estimation Python scripts for performing stereo depth estimation using the HITNET Tensorflow model from Google Research. Stereo de

Ibai Gorordo 76 Jan 2, 2023
Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite.

TFLite-HITNET-Stereo-depth-estimation Python scripts form performing stereo depth estimation using the HITNET model in Tensorflow Lite. Stereo depth e

Ibai Gorordo 22 Oct 20, 2022
Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in ONNX

ONNX msg_chn_wacv20 depth completion Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20 model in

Ibai Gorordo 19 Oct 22, 2022
Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

Ibai Gorordo 99 Dec 31, 2022
Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model in Tensorflow Lite.

TFLite-msg_chn_wacv20-depth-completion Python script for performing depth completion from sparse depth and rgb images using the msg_chn_wacv20. model

Ibai Gorordo 2 Oct 4, 2021