Tensorflow implementation of DeepLabv2

Related tags

Deep Learning TF-deeplab
Overview

TF-deeplab

This is a Tensorflow implementation of DeepLab, compatible with Tensorflow 1.2.1.

Currently it supports both training and testing the ResNet 101 version by converting the caffemodel provided by Jay.

Note that the current version is not multi-scale, i.e. only uses the original resolution branch and discarding all layers of 0.5 and 0.75 resolution.

The caffemodel2npy.py is modified from here, and the deeplab_model.py is modified from here.

Example Usage

  • Download the prototxt and caffemodel provided by Jay
  • Convert caffemodel to npy file
python caffemodel2npy.py deploy.prototxt ../deeplab/ResNet101/init.caffemodel ./model/ResNet101_init.npy
python caffemodel2npy.py deploy.prototxt ../deeplab/ResNet101/train_iter_20000.caffemodel ./model/ResNet101_train.npy
python caffemodel2npy.py deploy.prototxt ../deeplab/ResNet101/train2_iter_20000.caffemodel ./model/ResNet101_train2.npy
  • Convert npy file to tfmodel
python npy2tfmodel.py 0 ./model/ResNet101_init.npy ./model/ResNet101_init.tfmodel
python npy2tfmodel.py 0 ./model/ResNet101_train.npy ./model/ResNet101_train.tfmodel
python npy2tfmodel.py 0 ./model/ResNet101_train2.npy ./model/ResNet101_train2.tfmodel
  • Test on a single image
python deeplab_main.py 0 single
  • Test on the PASCAL VOC2012 validation set (you will also want to look at the matlab folder and run EvalSegResults.m after you run the following command)
python deeplab_main.py 0 test
  • To train on the PASCAL VOC2012 train_aug, run
python deeplab_main.py 0 train

Performance

The converted DeepLab ResNet 101 model achieves mean IOU of 73.296% on the validation set of PASCAL VOC2012. Again, this is only with the original resolution branch, which is likely to be the reason for the performance gap (according to the paper this number should be around 75%).

TODO

  • Incorporating 0.5 and 0.75 resolution
Comments
  • batch normalization initialization

    batch normalization initialization

    In the model, you initialize factor as:

    factor = tf.get_variable( 'factor', 1, tf.float32, initializer=tf.constant_initializer(0.0, tf.float32), trainable=False)

    which makes everything nan on first pass

    opened by eambutu 3
  • Converting caffemodel to npy file error

    Converting caffemodel to npy file error

    When I ran python caffemodel2npy.py deploy.prototxt ../deeplab/ResNet101/init.caffemodel/model/ResNet101_init.npy, an error occurred as the following.

    WARNING: Logging before InitGoogleLogging() is written to STDERR W0725 18:40:56.169318 9626 _caffe.cpp:122] DEPRECATION WARNING - deprecated use of Python interface W0725 18:40:56.169384 9626 _caffe.cpp:123] Use this instead (with the named "weights" parameter): W0725 18:40:56.169395 9626 _caffe.cpp:125] Net('deploy.prototxt', 1, weights='../deeplab/ResNet101/init.caffemodel') [libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 15:16: Message type "caffe.LayerParameter" has no field named "interp_param". F0725 18:40:56.173895 9626 upgrade_proto.cpp:79] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: deploy.prototxt *** Check failure stack trace: *** Abort (core dumped)

    What can the problem possibly be?

    opened by m-Just 2
  • maybe there is a softmax bug

    maybe there is a softmax bug

    the operation already include a softmax function. tf.nn.sparse_softmax_cross_entropy_with_logits.

    and before sparse_softmax_cross_entropy_with_logits you also have a softmax, which might slight influence the result.

    opened by dongzhuoyao 1
  • polynomial decay learning rate

    polynomial decay learning rate

    https://github.com/chenxi116/TF-deeplab/blob/439c872b3d9178d4f28731b25e635c01882193b5/deeplab_model.py#L163

    The default end_learning_rate is 0.0001 (https://www.tensorflow.org/api_docs/python/tf/train/polynomial_decay). However, according to https://stackoverflow.com/questions/30033096/what-is-lr-policy-in-caffe/30045244, the default caffe behavior is end_learning_rate=0.

    opened by eambutu 0
  • iter_size in original caffe implementation

    iter_size in original caffe implementation

    In the solver.prototxt of the original implementation, iter_size: 10 specifies that gradient updates only happen every 10 batches. However, the training script here doesnt do such a thing

    opened by eambutu 1
Owner
Chenxi Liu
Ph.D. Student in Computer Science
Chenxi Liu
GitHub
TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

null 2.6k Jan 4, 2023
Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Peter Lin 6.5k Jan 4, 2023
Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting (RVM) English | 中文 Official repository for the paper Robust High-Resolution Video Matting with Temporal Guidance. RVM is specific

flow-dev 2 Aug 21, 2022
Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow

xRBM Library Implementation of Restricted Boltzmann Machine (RBM) and its variants in Tensorflow Installation Using pip: pip install xrbm Examples Tut

Omid Alemi 55 Dec 29, 2022
Functional TensorFlow Implementation of Singular Value Decomposition for paper Fast Graph Learning

tf-fsvd TensorFlow Implementation of Functional Singular Value Decomposition for paper Fast Graph Learning with Unique Optimal Solutions Cite If you f

Sami Abu-El-Haija 14 Nov 25, 2021
StyleGAN2 - Official TensorFlow Implementation

StyleGAN2 - Official TensorFlow Implementation

NVIDIA Research Projects 10.1k Dec 28, 2022
An efficient and effective learning to rank algorithm by mining information across ranking candidates. This repository contains the tensorflow implementation of SERank model. The code is developed based on TF-Ranking.

SERank An efficient and effective learning to rank algorithm by mining information across ranking candidates. This repository contains the tensorflow

Zhihu 44 Oct 20, 2022
Implementation of Perceiver, General Perception with Iterative Attention in TensorFlow

Perceiver This Python package implements Perceiver: General Perception with Iterative Attention by Andrew Jaegle in TensorFlow. This model builds on t

Rishit Dagli 84 Oct 15, 2022
Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow.

Denoised-Smoothing-TF Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow. Denoised Smoothing is

Sayak Paul 19 Dec 11, 2022
Unofficial Implementation of MLP-Mixer in TensorFlow

mlp-mixer-tf Unofficial Implementation of MLP-Mixer [abs, pdf] in TensorFlow. Note: This project may have some bugs in it. I'm still learning how to i

Rishabh Anand 24 Mar 23, 2022
Tensorflow implementation for Self-supervised Graph Learning for Recommendation

If the compilation is successful, the evaluator of cpp implementation will be called automatically. Otherwise, the evaluator of python implementation will be called.

null 152 Jan 7, 2023
Minimal implementation of PAWS (https://arxiv.org/abs/2104.13963) in TensorFlow.

PAWS-TF ?? Implementation of Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples (PAWS)

Sayak Paul 43 Jan 8, 2023
Unofficial TensorFlow implementation of the Keyword Spotting Transformer model

Keyword Spotting Transformer This is the unofficial TensorFlow implementation of the Keyword Spotting Transformer model. This model is used to train o

Intelligent Machines Limited 8 May 11, 2022
A tensorflow implementation of GCN-LPA

GCN-LPA This repository is the implementation of GCN-LPA (arXiv): Unifying Graph Convolutional Neural Networks and Label Propagation Hongwei Wang, Jur

Hongwei Wang 83 Nov 28, 2022
Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Detection"

M-LSD: Towards Light-weight and Real-time Line Segment Detection Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line

NAVER/LINE Vision 357 Jan 4, 2023
Unofficial TensorFlow implementation of Protein Interface Prediction using Graph Convolutional Networks.

[TensorFlow] Protein Interface Prediction using Graph Convolutional Networks Unofficial TensorFlow implementation of Protein Interface Prediction usin

YeongHyeon Park 9 Oct 25, 2022
Tensorflow implementation of MIRNet for Low-light image enhancement

MIRNet Tensorflow implementation of the MIRNet architecture as proposed by Learning Enriched Features for Real Image Restoration and Enhancement. Lanu

Soumik Rakshit 91 Jan 6, 2023
Tensorflow python implementation of "Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos"

Learning High Fidelity Depths of Dressed Humans by Watching Social Media Dance Videos This repository is the official tensorflow python implementation

Yasamin Jafarian 287 Jan 6, 2023
Tensorflow implementation of Swin Transformer model.

Swin Transformer (Tensorflow) Tensorflow reimplementation of Swin Transformer model. Based on Official Pytorch implementation. Requirements tensorflow

null 167 Jan 8, 2023