SegNet-like Autoencoders in TensorFlow

Related tags

Deep Learning computer-vision deep-learning tensorflow segmentation convolutional-autoencoder segnet
Overview

SegNet

SegNet is a TensorFlow implementation of the segmentation network proposed by Kendall et al., with cool features like strided deconvolution, a minified architecture and more.

Configuration

Create a config.py file, containing color maps, working dataset and other options.

autoencoder = 'segnet'
colors = {
  'segnet-32': [
    [64, 128, 64],   # Animal
    [192, 0, 128],   # Archway
    [0, 128, 192],   # Bicyclist
    [0, 128, 64],    # Bridge
    [128, 0, 0],     # Building
    [64, 0, 128],    # Car
    [64, 0, 192],    # CartLuggagePram
    [192, 128, 64],  # Child
    [192, 192, 128], # Column_Pole
    [64, 64, 128],   # Fence
    [128, 0, 192],   # LaneMkgsDriv
    [192, 0, 64],    # LaneMkgsNonDriv
    [128, 128, 64],  # Misc_Text
    [192, 0, 192],   # MotorcycleScooter
    [128, 64, 64],   # OtherMoving
    [64, 192, 128],  # ParkingBlock
    [64, 64, 0],     # Pedestrian
    [128, 64, 128],  # Road
    [128, 128, 192], # RoadShoulder
    [0, 0, 192],     # Sidewalk
    [192, 128, 128], # SignSymbol
    [128, 128, 128], # Sky
    [64, 128, 192],  # SUVPickupTruck
    [0, 0, 64],      # TrafficCone
    [0, 64, 64],     # TrafficLight
    [192, 64, 128],  # Train
    [128, 128, 0],   # Tree
    [192, 128, 192], # Truck_Bus
    [64, 0, 64],     # Tunnel
    [192, 192, 0],   # VegetationMisc
    [0, 0, 0],       # Void
    [64, 192, 0]     # Wall
  ]
}
gpu_memory_fraction = 0.7
strided = True
working_dataset = 'segnet-32'

Two kinds of architectures are supported at the moment: the original SegNet Encoder-Decoder (segnet), and a smaller version of the same (mini), which can be used for simpler segmentation problems. I suggest to use strided = True for faster and more reliable results.

The dataset_name needs to match the data directories you create in your input folder. You can use segnet-32 and segnet-13 to replicate the aforementioned Kendall et al. experiments.

Train and test

Generate your TFRecords using tfrecorder.py. In order to do so, put your PNG images in a raw folder, as follows:

input/
    raw/
        train/
        train-labels/
        test/
        test-labels/

Once you have your TFRecords, train SegNet with python src/train.py. Analogously, test it with python src/test.py.

Comments
  • training issue: train-labels / test-labels format

    training issue: train-labels / test-labels format

    Hello,

    I'm getting the following error when I train my model after preparing my TFRecords files: OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 5, current size 0) [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]

    All my images are 224x224 .PNGs, so I'm assuming that there's either a formatting issue with my training and test labelled images, or an issue with me using me using the CPU version of TensorFlow.

    If the issue isn't from using the CPU version of TensorFlow, what sort of transforming is needed for the image annotations before using the tfrecorder to convert them? Does it involve converting the annotated images into color-mapped images using the colors from the config.py file?

    Thanks!!

    opened by DanielMolina 5
  • Training issue

    Training issue

    Hi @andreaazzini I'm trying to use your code and I'm getting issues related to the Queue Shuffling during training.

    Do you know what should I do to proceed?

    OutOfRangeError (see above for traceback): RandomShuffleQueue '_2_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 12, current size 0)
	 [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/random_shuffle_queue, shuffle_batch/n)]]
    opened by Giounona 5
  • input and dataset

    input and dataset

    Hi,

    I just found your project. Actually, it looks pretty cool/good.

    I have some questions to ask you. One of them is if you could show me the structure of your input folder. Another one is related to that folder too. I really can't find the segnet-32 dataset, do you still have it? or do you have any dataset avaidable?

    I've also seen you've been updating your projecto to tensorflow v1.0. Is the proyect fully updated or at least does the current version work fine?

    I would be very gratefull if you could help me.

    Thanks!

    opened by Shathe 3
  • OutOfRangeError revisited

    OutOfRangeError revisited

    hi, i experience the same issue, namely -

    tensorflow.python.framework.errors_impl.OutOfRangeError: RandomShuffleQueue '...' is closed and has insufficient elements (requested 12, current size 0) File "src/train.py", line 25, in train images, labels = inputs(FLAGS.batch, FLAGS.train, FLAGS.train_labels) File "src/inputs.py", line 29, in inputs shuffle_batch

    but i fail to understand what is meant by "correct resolution" in the related closed issue #10 . thanks!

    opened by michath 1
  • OutOfRangeError

    OutOfRangeError

    Hi I'm training your net with camvid dataset. I followed your instructions but I get this error:

    OutOfRangeError (see above for traceback): RandomShuffleQueue '_3_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 2, current size 0)

    I think its not finding the training images which I did with your script. Could you please explain what you mean with this:

    The dataset_name needs to match the data directories you create in your input folder. You can use segnet-32 and segnet-13 to replicate the aforementioned Kendall et al. experiments.

    cheers

    opened by eflavio 1
  • Introduce image encoding for labels and independent train and test.

    Introduce image encoding for labels and independent train and test.

    • [x] datachef.py becomes TFRecorder.py, an even faster and more specialized TFRecords encoder
    • [x] Datasets organization
    • [x] Configuration file
    • [x] Independent train and test files
    • [x] Code hierarchically structured into modules
    opened by andreaazzini 0
  • Error while training

    Error while training

    when I try to train the model I obtain this error could somebody help me?

    Saving results to input Launching 1 threads for spacings: [[0, 71]] 2018-02-13 15:15:23.414452: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX Exception in thread Thread-1: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "tfrecorder.py", line 96, in _process_image_files_batch image_buffer, height, width = _process_image(filename, coder) File "tfrecorder.py", line 53, in _process_image image_data = f.read() File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 126, in read pywrap_tensorflow.ReadFromStream(self._read_buf, length, status)) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 94, in _prepare_value return compat.as_str_any(val) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 106, in as_str_any return as_str(value) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 84, in as_text return bytes_or_text.decode(encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

    2018-02-13 15:15:24.444012: Finished writing all 71 images in data set. Launching 1 threads for spacings: [[0, 39]] Exception in thread Thread-2: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "tfrecorder.py", line 96, in _process_image_files_batch image_buffer, height, width = _process_image(filename, coder) File "tfrecorder.py", line 53, in _process_image image_data = f.read() File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 126, in read pywrap_tensorflow.ReadFromStream(self._read_buf, length, status)) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 94, in _prepare_value return compat.as_str_any(val) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 106, in as_str_any return as_str(value) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 84, in as_text return bytes_or_text.decode(encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

    2018-02-13 15:15:25.452242: Finished writing all 39 images in data set. Launching 1 threads for spacings: [[0, 76]] Exception in thread Thread-3: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "tfrecorder.py", line 96, in _process_image_files_batch image_buffer, height, width = _process_image(filename, coder) File "tfrecorder.py", line 53, in _process_image image_data = f.read() File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 126, in read pywrap_tensorflow.ReadFromStream(self._read_buf, length, status)) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 94, in _prepare_value return compat.as_str_any(val) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 106, in as_str_any return as_str(value) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 84, in as_text return bytes_or_text.decode(encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

    2018-02-13 15:15:26.460194: Finished writing all 76 images in data set. Launching 1 threads for spacings: [[0, 82]] Exception in thread Thread-4: Traceback (most recent call last): File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner self.run() File "/usr/lib/python3.5/threading.py", line 862, in run self._target(*self._args, **self._kwargs) File "tfrecorder.py", line 96, in _process_image_files_batch image_buffer, height, width = _process_image(filename, coder) File "tfrecorder.py", line 53, in _process_image image_data = f.read() File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 126, in read pywrap_tensorflow.ReadFromStream(self._read_buf, length, status)) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/lib/io/file_io.py", line 94, in _prepare_value return compat.as_str_any(val) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 106, in as_str_any return as_str(value) File "/home/cedriq/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 84, in as_text return bytes_or_text.decode(encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

    2018-02-13 15:15:27.467536: Finished writing all 82 images in data set.

    opened by fotsing365 1
  • VGG16 Weight Initialisation

    VGG16 Weight Initialisation

    Hi, I notice in previous commits you have functionality to initialise the encoder weights from a pre-trained VGG16, but this does not appear to be in the latest version. Why has this been removed?

    opened by msinto93 0
  • final outputs

    final outputs

    Thanks a lot for sharing your great work. Could you please, add some parts to describe the final outputs (i.e. the predicted labels) of the test and training images? Best

    opened by sevamoo 0
  • Varaibles init errors

    Varaibles init errors

    I get the error RandomShuffleQueue '_3_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 12, current size 0) in train.py train().inputs() call. If I move the init variables statement to before inputs() is called, pool, unpool variables are initialized properly, but during AdamOptimizer init, I get the error for uninitialized variables beta1.

    Can you please help fix the variables initialization? I'm on Win 10, python 3.6 x64, TensorFlow 1.2.0

    opened by divyanshugrover 2
  • The unpool is not standard right ?

    The unpool is not standard right ?

    Hello,

    from the code I have read in your project, it the unpool operation is just an image resize. So basically, this network is not really a segnet network (at least not in a 'canonical' way from the paper). Do you plan to use the locations of the max-pool layers to implement the standard unpool op ?

    opened by DrM175u 2
Owner
Andrea Azzini
⚡
Andrea Azzini
GitHub
Implementation of experiments in the paper Clockwork Variational Autoencoders (project website) using JAX and Flax

Clockwork VAEs in JAX/Flax Implementation of experiments in the paper Clockwork Variational Autoencoders (project website) using JAX and Flax, ported

Julius Kunze 26 Oct 5, 2022
Official implementation of the paper "AAVAE: Augmentation-AugmentedVariational Autoencoders"

AAVAE Official implementation of the paper "AAVAE: Augmentation-AugmentedVariational Autoencoders" Abstract Recent methods for self-supervised learnin

Grid AI Labs 48 Dec 12, 2022
Code for the paper "Adversarially Regularized Autoencoders (ICML 2018)" by Zhao, Kim, Zhang, Rush and LeCun

ARAE Code for the paper "Adversarially Regularized Autoencoders (ICML 2018)" by Zhao, Kim, Zhang, Rush and LeCun https://arxiv.org/abs/1706.04223 Disc

Junbo (Jake) Zhao 399 Jan 2, 2023
Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders

Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders

null 1 Oct 11, 2021
Data Augmentation with Variational Autoencoders

Documentation Pyraug This library provides a way to perform Data Augmentation using Variational Autoencoders in a reliable way even in challenging con

null 112 Nov 30, 2022
PyTorch Autoencoders - Implementing a Variational Autoencoder (VAE) Series in Pytorch.

PyTorch Autoencoders Implementing a Variational Autoencoder (VAE) Series in Pytorch. Inspired by this repository Model List check model paper conferen

Subin An 8 Nov 21, 2022
Predicting lncRNA–protein interactions based on graph autoencoders and collaborative training

Predicting lncRNA–protein interactions based on graph autoencoders and collaborative training Code for our paper "Predicting lncRNA–protein interactio

zhanglabNKU 1 Nov 29, 2022
Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners

Unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners This repository is built upon BEiT, thanks very much! Now, we on

Zhiliang Peng 2.3k Jan 4, 2023
PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners for self-supervised ViT.

MAE for Self-supervised ViT Introduction This is an unofficial PyTorch implementation of Masked Autoencoders Are Scalable Vision Learners for self-sup

null 36 Oct 30, 2022
An pytorch implementation of Masked Autoencoders Are Scalable Vision Learners

An pytorch implementation of Masked Autoencoders Are Scalable Vision Learners This is a coarse version for MAE, only make the pretrain model, the fine

FlyEgle 214 Dec 29, 2022
A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks

A framework that constructs deep neural networks, autoencoders, logistic regressors, and linear networks without the use of any outside machine learning libraries - all from scratch.

Kordel K. France 2 Nov 14, 2022
Autoencoders pretraining using clustering

Autoencoders pretraining using clustering

IITiS PAN 2 Dec 16, 2021
Re-implememtation of MAE (Masked Autoencoders Are Scalable Vision Learners) using PyTorch.

mae-repo PyTorch re-implememtation of "masked autoencoders are scalable vision learners". In this repo, it heavily borrows codes from codebase https:/

Peng Qiao 1 Dec 14, 2021
ConvMAE: Masked Convolution Meets Masked Autoencoders

ConvMAE ConvMAE: Masked Convolution Meets Masked Autoencoders Peng Gao1, Teli Ma1, Hongsheng Li2, Jifeng Dai3, Yu Qiao1, 1 Shanghai AI Laboratory, 2 M

Alpha VL Team of Shanghai AI Lab 345 Jan 8, 2023
Code and pre-trained models for MultiMAE: Multi-modal Multi-task Masked Autoencoders

MultiMAE: Multi-modal Multi-task Masked Autoencoders Roman Bachmann*, David Mizrahi*, Andrei Atanov, Amir Zamir Website | arXiv | BibTeX Official PyTo

Visual Intelligence & Learning Lab, Swiss Federal Institute of Technology (EPFL) 385 Jan 6, 2023
VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training [Arxiv] VideoMAE: Masked Autoencoders are Data-Efficient Learne

Multimedia Computing Group, Nanjing University 697 Jan 7, 2023
Hand gesture recognition based whiteboard that allows you to write on live webcam. This is the first version and has features like 4 different colors, eraser and a recording option that records your session and saves it in a "recordings" folder. Use index finger to draw and two or more fingers to move around and select items. Future version will contain more functionalities like changeable thickness, color palette, integration with zoom and google meet etc.

hand-write Hand gesture recognition based whiteboard that allows you to write on live webcam. This is the first version and has features like 4 differ

null 27 Dec 16, 2022
Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy.

Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy. Now with tensorflow 1.0 support. Evaluation usa

Marcel R. 349 Aug 6, 2022
TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

null 2.6k Jan 4, 2023