Deep Watershed Transform for Instance Segmentation

Related tags

Deep Learning dwt
Overview

Deep Watershed Transform

Performs instance level segmentation detailed in the following paper:

Min Bai and Raquel Urtasun, Deep Watershed Transformation for Instance Segmentation, in CVPR 2017. Accessible at https://arxiv.org/abs/1611.08303.

This page is still under construction.

Dependencies

Developed and tested on Ubuntu 14.04 and 16.04.

  1. TensorFlow www.tensorflow.org
  2. Numpy, Scipy, and Skimage (sudo apt-get install python-numpy python-scipy python-skimage)

Inputs

  1. Cityscapes images (www.cityscapes-dataset.com).
  2. Semantic Segmentation for input images. In our case, we used the output from PSPNet (by H. Zhao et al. https://github.com/hszhao/PSPNet). These are uint8 images with pixel-wise semantic labels encoded with 'trainIDs' defined by Cityscapes. For more information, visit https://github.com/mcordts/cityscapesScripts/blob/master/cityscapesscripts/helpers/labels.py

Outputs

The model produces pixel-wise instance labels as a uint16 image with the same formatting as the Cityscapes instance segmentation challenge ground truth. In particular, each pixel is labeled as 'id' * 1000 + instance_id, where 'id' is as defined by Cityscapes (for more information, consult labels.py in the above link), and instance_id is an integer indexing the object instance.

Testing the Model

  1. Clone repository into dwt/.
  2. Download the model from www.cs.toronto.edu/~mbai/dwt_cityscapes_pspnet.mat and place into the "dwt/model" directory.
  3. run "cd E2E"
  4. run "python main.py"
  5. The results will be available in "dwt/example/output".

Training the Model

  1. Will be available soon.
Comments
  • issue when generating ground truth for direction map and WTN

    issue when generating ground truth for direction map and WTN

    I tried to run the batch_generate_cityscapes.m to generate the ground truth mat. I encountered the following error in generate_GT_cityscapes_unified.m Error using repmat Too many input arguments.

    Error in generate_GT_cityscapes_unified (line 13) dir_map = repmat(depth_map, 1, 1, 2); ps. I used Matlab 2013a Can someone please tell me what repmat(depth_map, 1, 1, 2); is intended to do? Also, could someone tell me what 'depth_map', 'dir_map', 'weight_map', 'edge_map' represent? thanks. I suppose dir_map is direction of distance transform.

    opened by bill2239 5
  • Can You Share the Example Floder

    Can You Share the Example Floder

    Dear author: I try to compile the project you share, however there is no "Example" floder. Could you share this floder or reply me the content in the floder? Thank you!

    opened by LiuYqi 1
  • ValueError: Shape must be rank 4 but is rank 0 for 'model_builder/direction/conv1_1/Conv2D' (op: 'Conv2D') with input shapes: [], [3,3,4,64].

    ValueError: Shape must be rank 4 but is rank 0 for 'model_builder/direction/conv1_1/Conv2D' (op: 'Conv2D') with input shapes: [], [3,3,4,64].

    There is something wrong when I run the main.py. It seems like when building the first conv layer the builder could not get the shape of input data (the placeholders' shape is just [ ] ). What should I do to run the code? Thanks.

    opened by guanxiongsun 1
  • InvalidArgumentError: Incompatible shapes: [1,512,1024,3] vs. [1,512,1024,4,1]

    InvalidArgumentError: Incompatible shapes: [1,512,1024,3] vs. [1,512,1024,4,1]

    Traceback (most recent call last): File "main.py", line 69, in outputSavePath="../example/output") File "/usr/qiaodong/dwt/E2E/forward.py", line 35, in forward_model keepProb: 1.0}) File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 789, in run run_metadata_ptr) File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 997, in _run feed_dict_string, options, run_metadata) File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run target_list, options, run_metadata) File "/usr/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [1,512,1024,3] vs. [1,512,1024,4,1] [[Node: model_builder/mul = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_Placeholder_0_0, model_builder/ExpandDims)]]

    Caused by op u'model_builder/mul', defined at: File "main.py", line 69, in outputSavePath="../example/output") File "/usr/qiaodong/dwt/E2E/forward.py", line 19, in forward_model model.build(tfBatchImages, tfBatchSS, tfBatchSSMask, keepProb=keepProb) File "/usr/qiaodong/dwt/E2E/e2e_model.py", line 21, in build inputData = inputData * ss File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 838, in binary_op_wrapper return func(x, y, name=name) File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 1061, in _mul_dispatch return gen_math_ops._mul(x, y, name=name) File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1377, in _mul result = _op_def_lib.apply_op("Mul", x=x, y=y, name=name) File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op op_def=op_def) File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op original_op=self._default_original_op, op_def=op_def) File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1269, in init self._traceback = _extract_stack()

    InvalidArgumentError (see above for traceback): Incompatible shapes: [1,512,1024,3] vs. [1,512,1024,4,1] [[Node: model_builder/mul = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_Placeholder_0_0, model_builder/ExpandDims)]]

    opened by qiaoD 0
  • TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

    TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

    When running main.py as you described in the readme, I get the following error:

    Traceback (most recent call last): File "main.py", line 69, in outputSavePath="../example/output") File "/home/mehdi/Desktop/deeplearning/dwt/E2E/forward.py", line 19, in forward_model model.build(tfBatchImages, tfBatchSS, tfBatchSSMask, keepProb=keepProb) File "/home/mehdi/Desktop/deeplearning/dwt/E2E/e2e_model.py", line 23, in build inputData = tf.concat(3, [inputData, tf.expand_dims(ssMask,-1)]) File "/home/mehdi/environments/tensorflow-py2.7/local/lib/python2.7/site-packages/tensorflow/python/ops/array_ops.py", line 1111, in concat dtype=dtypes.int32).get_shape().assert_is_compatible_with( File "/home/mehdi/environments/tensorflow-py2.7/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 998, in convert_to_tensor as_ref=False) File "/home/mehdi/environments/tensorflow-py2.7/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1094, in internal_convert_to_tensor ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) File "/home/mehdi/environments/tensorflow-py2.7/local/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 217, in _constant_tensor_conversion_function return constant(v, dtype=dtype, name=name) File "/home/mehdi/environments/tensorflow-py2.7/local/lib/python2.7/site-packages/tensorflow/python/framework/constant_op.py", line 196, in constant value, dtype=dtype, shape=shape, verify_shape=verify_shape)) File "/home/mehdi/environments/tensorflow-py2.7/local/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 436, in make_tensor_proto _AssertCompatible(values, dtype) File "/home/mehdi/environments/tensorflow-py2.7/local/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 347, in _AssertCompatible (dtype.name, repr(mismatch), type(mismatch).name)) TypeError: Expected int32, got list containing Tensors of type '_Message' instead.

    Am I missing something?

    opened by mehditlili 4
Owner
null
GitHub
Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation

Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation This paper has been accepted and early accessed

Yun Liu 39 Sep 20, 2022
TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

yifan liu 147 Dec 3, 2022
[ArXiv 2021] Data-Efficient Instance Generation from Instance Discrimination

InsGen - Data-Efficient Instance Generation from Instance Discrimination Data-Efficient Instance Generation from Instance Discrimination Ceyuan Yang,

GenForce: May Generative Force Be with You 93 Dec 25, 2022
Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [CVPR 2021]

Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [BCNet, CVPR 2021] This is the official pytorch implementation of BCNet built on

Lei Ke 434 Dec 1, 2022
Numbering permanent and deciduous teeth via deep instance segmentation in panoramic X-rays

Numbering permanent and deciduous teeth via deep instance segmentation in panoramic X-rays In this repo, you will find the instructions on how to requ

Intelligent Vision Research Lab 4 Jul 21, 2022
Style transfer, deep learning, feature transform

FastPhotoStyle License Copyright (C) 2018 NVIDIA Corporation. All rights reserved. Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons

NVIDIA Corporation 10.9k Jan 2, 2023
Classifying audio using Wavelet transform and deep learning

Audio Classification using Wavelet Transform and Deep Learning A step-by-step tutorial to classify audio signals using continuous wavelet transform (C

Aditya Dutt 17 Nov 29, 2022
YolactEdge: Real-time Instance Segmentation on the Edge

YolactEdge, the first competitive instance segmentation approach that runs on small edge devices at real-time speeds. Specifically, YolactEdge runs at up to 30.8 FPS on a Jetson AGX Xavier (and 172.7 FPS on an RTX 2080 Ti) with a ResNet-101 backbone on 550x550 resolution images.

Haotian Liu 1.1k Jan 6, 2023
the code used for the preprint Embedding-based Instance Segmentation of Microscopy Images.

EmbedSeg Introduction This repository hosts the version of the code used for the preprint Embedding-based Instance Segmentation of Microscopy Images.

JugLab 88 Dec 25, 2022
Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation

Unseen Object Clustering: Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation Introduction In this work, we propose a new method

NVIDIA Research Projects 132 Dec 13, 2022
[CVPR2021 Oral] End-to-End Video Instance Segmentation with Transformers

VisTR: End-to-End Video Instance Segmentation with Transformers This is the official implementation of the VisTR paper: Installation We provide instru

Yuqing Wang 687 Jan 7, 2023
git《USD-Seg:Learning Universal Shape Dictionary for Realtime Instance Segmentation》(2020) GitHub: [fig2]

USD-Seg This project is an implement of paper USD-Seg:Learning Universal Shape Dictionary for Realtime Instance Segmentation, based on FCOS detector f

Ruolin Ye 80 Nov 28, 2022
This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Swin Transformer for Object Detection This repo contains the supported code and configuration files to reproduce object detection results of Swin Tran

Swin Transformer 1.4k Dec 30, 2022
Implementation of DropLoss for Long-Tail Instance Segmentation in Pytorch

[AAAI 2021]DropLoss for Long-Tail Instance Segmentation [AAAI 2021] DropLoss for Long-Tail Instance Segmentation Ting-I Hsieh*, Esther Robb*, Hwann-Tz

Tim 37 Dec 2, 2022
Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch.

Faster R-CNN and Mask R-CNN in PyTorch 1.0 maskrcnn-benchmark has been deprecated. Please see detectron2, which includes implementations for all model

Facebook Research 9k Jan 4, 2023
Object detection and instance segmentation toolkit based on PaddlePaddle.

Object detection and instance segmentation toolkit based on PaddlePaddle.

null 9.3k Jan 2, 2023
PyTorch code for the paper "FIERY: Future Instance Segmentation in Bird's-Eye view from Surround Monocular Cameras"

FIERY This is the PyTorch implementation for inference and training of the future prediction bird's-eye view network as described in: FIERY: Future In

Wayve 406 Dec 24, 2022
ISTR: End-to-End Instance Segmentation with Transformers (https://arxiv.org/abs/2105.00637)

This is the project page for the paper: ISTR: End-to-End Instance Segmentation via Transformers, Jie Hu, Liujuan Cao, Yao Lu, ShengChuan Zhang, Yan Wa

Jie Hu 182 Dec 19, 2022
(IEEE TIP 2021) Regularized Densely-connected Pyramid Network for Salient Instance Segmentation

RDPNet IEEE TIP 2021: Regularized Densely-connected Pyramid Network for Salient Instance Segmentation PyTorch training and testing code are available.

Yu-Huan Wu 41 Oct 21, 2022