DeconvNet : Learning Deconvolution Network for Semantic Segmentation

Related tags

Deep Learning DeconvNet
Overview

DeconvNet: Learning Deconvolution Network for Semantic Segmentation

Created by Hyeonwoo Noh, Seunghoon Hong and Bohyung Han at POSTECH

Acknowledgements: Thanks to Yangqing Jia and the BVLC team for creating Caffe.

Introduction

DeconvNet is state-of-the-art semantic segmentation system that combines bottom-up region proposals with multi-layer decovolution network.

Detailed description of the system will be provided by our technical report [arXiv tech report] http://arxiv.org/abs/1505.04366

Citation

If you're using this code in a publication, please cite our papers.

@article{noh2015learning,
  title={Learning Deconvolution Network for Semantic Segmentation},
  author={Noh, Hyeonwoo and Hong, Seunghoon and Han, Bohyung},
  journal={arXiv preprint arXiv:1505.04366},
  year={2015}
}

Pre-trained Model

If you need model definition and pre-trained model only, you can download them from following location: 0. caffe for DeconvNet: https://github.com/HyeonwooNoh/caffe 0. DeconvNet model definition: http://cvlab.postech.ac.kr/research/deconvnet/model/DeconvNet/DeconvNet_inference_deploy.prototxt 0. Pre-trained DeconvNet weight: http://cvlab.postech.ac.kr/research/deconvnet/model/DeconvNet/DeconvNet_trainval_inference.caffemodel

Licence

This software is being made available for research purpose only. Check LICENSE file for details.

System Requirements

This software is tested on Ubuntu 14.04 LTS (64bit).

Prerequisites 0. MATLAB (tested with 2014b on 64-bit Linux) 0. prerequisites for caffe(http://caffe.berkeleyvision.org/installation.html#prequequisites)

Installing DeconvNet

By running "setup.sh" you can download all the necessary file for training and inference include: 0. caffe: you need modified version of caffe which support DeconvNet - https://github.com/HyeonwooNoh/caffe.git 0. data: data used for training stage 1 and 2 0. model: caffemodel of trained DeconvNet and other caffemodels required for training

Training DeconvNet

Training scripts are included in ./training/ directory

To train DeconvNet you can simply run following scripts in order: 0. 001_start_train.sh : script for first stage training 0. 002_start_train.sh : script for second stage training 0. 003_start_make_bn_layer_testable : script converting trained DeconvNet with bn layer to inference mode

Inference EDeconvNet+CRF

Run run_demo.m to reproduce EDeconvNet+CRF results on VOC2012 test data.

This script will generated EDeconvNet+CRF results through following steps: 0. run FCN-8s and cache the score [cache_FCN8s_results.m] 0. generate DeconvNet score and apply ensemble with FCN-8s score, post processing with densecrf [generate_EDeconvNet_CRF_results.m]

EDeconvNet+CRF obtains 72.5 mean I/U on PASCAL VOC 2012 Test

External dependencies [can be downloaded by running "setup.sh" script] 0. FCN-8s model and weight file [https://github.com/BVLC/caffe/wiki/Model-Zoo] 0. densecrf with matlab wrapper [https://github.com/johannesu/meanfield-matlab.git] 0. cached proposal bounding boxes extracted with edgebox object proposal [https://github.com/pdollar/edges]

Comments
  • error at cudaSuccess?

    error at cudaSuccess?

    Hi,

    I'm trying to train DeconvNet and run 001_start_train.sh ,but the error is:

    F0907 11:37:08.948395 5462 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory *** Check failure stack trace: *** @ 0x7fe9bc71da3d google::LogMessage::Fail() @ 0x7fe9bc721ed7 google::LogMessage::SendToLog() @ 0x7fe9bc71fd39 google::LogMessage::Flush() @ 0x7fe9bc72003d google::LogMessageFatal::~LogMessageFatal() @ 0x6a6f8e caffe::SyncedMemory::to_gpu() @ 0x6a6b20 caffe::SyncedMemory::mutable_gpu_data() @ 0x6aacfd caffe::Blob<>::mutable_gpu_data() @ 0x6d8b7d caffe::UnpoolingLayer<>::Forward_gpu() @ 0x577814 caffe::Layer<>::Forward() @ 0x58e419 caffe::Net<>::ForwardFromTo() @ 0x58e171 caffe::Net<>::ForwardPrefilled() @ 0x58e5a4 caffe::Net<>::Forward() @ 0x6b3962 caffe::Solver<>::Test() @ 0x6b376d caffe::Solver<>::TestAll() @ 0x6b2b51 caffe::Solver<>::Step() @ 0x6b268b caffe::Solver<>::Solve() @ 0x5738c1 train() @ 0x575903 main @ 0x36e9a1ed5d (unknown) @ 0x572df9 (unknown) ./start_train.sh: line 6: 5462 Aborted GLOG_log_dir=$LOGDIR $CAFFE train -solver $SOLVER -weights $WEIGHTS -gpu 0 cp: cannot stat `./snapshot/stage_1_train_iter_20000.caffemodel': No such file or directory

    I'm a new newbie in caffe , can u help me? thinks.

    opened by JanceeLee 2
  • Unknown enumeration value of

    Unknown enumeration value of "BN" for field "type".

    Use shell script can train net, but use python script to train ,I got this error " Unknown enumeration value of "BN" for field "type"."

    opened by WenbinYang123 0
  • Converting caffe model to Keras

    Converting caffe model to Keras

    I was trying to convert deconvnet caffe model to keras with tensorflow backend. I only transpose convolutional layer weights since I thought caffe and tensorflow use correlation. However I believe I've got stuck with probably batch normalization layers. I'm extracting bn weights but I don't know if they are required to be rotated or something else when using with keras batch normalization layer? Until now I've tried to use them by reshaping and setting to gamma and beta variables at keras batch norm layer.

    opened by hiiroo 0
  • What do these four numbers stand for?

    What do these four numbers stand for?

    There are four numbers in each row in the /DeconvNet/data/imagesets/stage_2_train_imgset/train.txt. What do these four numbers stand for? I try to train DeconvNet in my own dataset. So how to get these numbers? Thanks. /VOC2012_SEG_AUG/images/2010_002729.png /VOC2012_SEG_AUG/segmentations/2010_002729.png 53 188 289 424 /VOC2012_SEG_AUG/images/2010_004179.png /VOC2012_SEG_AUG/segmentations/2010_004179.png -52 -29 308 331 What do these four numbers -52 -29 308 331 stand for?

    opened by xujiayuxujiayu 1
  • DeconvNet fine tuning

    DeconvNet fine tuning

    Thanks for making your work available, deeply appreciated.

    I'm hoping to fine tune your pre-trained DeconvNet, but I noticed that stage 003 make BN layers testable. Hence I assume that I should not load 'DeconvNet_trainval_inference.caffemodel' straightaway and starts the training, but I'm not sure. Please let me know if my concern is valid, if it does, can you please upload the caffemodel after stage 002 training? Thanks a lot.

    opened by ckchng 4
  • Out of memory error while there is enough memory

    Out of memory error while there is enough memory

    So I setup everything, and when I run the training/001_start_train.sh, I get the out of memory error: Check failed: error == cudaSuccess (2 vs. 0) out of memory

    I have 6GB on my GPU, and constantly watching its memory-usage. I even tried setting the batch-size to 1 for both training and testing. None of these helped and I still receive this error.

    Since I get passed the network initialization and the initial memory size check, my guess is something is going on in the IMAGE_SEG_DATA layer. Are you loading all the data into the GPU memory at the very beginning to reduce disk access?

    opened by amir-abdi 0
  • Makefile:479: recipe for target '.build_release/src/caffe/layers/euclidean_loss_layer.o' failed

    Makefile:479: recipe for target '.build_release/src/caffe/layers/euclidean_loss_layer.o' failed

    I use the Makefile.config copying from my machine,which compiling successfully on the initial version of caffe ,but when it goes on the modified version,it made mistakes like this：

    Makefile:479: recipe for target '.build_release/src/caffe/layers/euclidean_loss_layer.o' failed

    make: *** [.build_release/src/caffe/layers/euclidean_loss_layer.o] Error 1 and when I got into that directory,using "find" command

    sheng@ML ~/DeconvNet/caffe $ find ~/DeconvNet/caffe/ -name euclidean_loss_layer.o sheng@ML ~/DeconvNet/caffe $

    I can not find the "euclidean_loss_layer.o",Any mistakes I make?

    opened by kilakila-heart 0
Owner
Hyeonwoo Noh
Hyeonwoo Noh
GitHub
Official Python implementation of the 'Sparse deconvolution'-v0.3.0

Sparse deconvolution Python v0.3.0 Official Python implementation of the 'Sparse deconvolution', and the CPU (NumPy) and GPU (CuPy) calculation backen

Weisong Zhao 23 Dec 28, 2022
Pmapper is a super-resolution and deconvolution toolkit for python 3.6+

pmapper pmapper is a super-resolution and deconvolution toolkit for python 3.6+. PMAP stands for Poisson Maximum A-Posteriori, a highly flexible and a

NASA Jet Propulsion Laboratory 8 Nov 6, 2022
Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation) Download Synthia dataset The model uses

null 32 Sep 21, 2022
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Jiwoon Ahn 337 Dec 15, 2022
Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP Abstract: We introduce a method that allows to automatically se

Daniil Pakhomov 134 Dec 19, 2022
TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

yifan liu 147 Dec 3, 2022
Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

ADE20k Semantic segmentation with MAE Getting started Install the mmsegmentation

null 97 Dec 17, 2022
(CVPR2021) DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation

DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation CVPR2021(oral) [arxiv] Requirements python3.7 pytorch==

W-zx-Y 85 Dec 7, 2022
Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation Created by Zeyu HU Introduction This work is based on our paper VMNet: Voxel-Mes

HU Zeyu 82 Dec 27, 2022
Adaptive Pyramid Context Network for Semantic Segmentation (APCNet CVPR'2019)

Adaptive Pyramid Context Network for Semantic Segmentation (APCNet CVPR'2019) Introduction Official implementation of Adaptive Pyramid Context Network

null 21 Nov 9, 2022
An extremely simple, intuitive, hardware-friendly, and well-performing network structure for LiDAR semantic segmentation on 2D range image. IROS21

FIDNet_SemanticKITTI Motivation Implementing complicated network modules with only one or two points improvement on hardware is tedious. So here we pr

YimingZhao 54 Dec 12, 2022
Train neural network for semantic segmentation (deep lab V3) with pytorch in less then 50 lines of code

Train neural network for semantic segmentation (deep lab V3) with pytorch in 50 lines of code Train net semantic segmentation net using Trans10K datas

null 17 Dec 19, 2022
DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation

DFFNet Paper DFFNet: An IoT-perceptive Dual Feature Fusion Network for General Real-time Semantic Segmentation. Xiangyan Tang, Wenxuan Tu, Keqiu Li, J

null 4 Sep 23, 2022
Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021)

Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation (CVPR 2021, official Pytorch implementatio

Microsoft 247 Dec 25, 2022
Official code of CVPR 2021's PLOP: Learning without Forgetting for Continual Semantic Segmentation

PLOP: Learning without Forgetting for Continual Semantic Segmentation This repository contains all of our code. It is a modified version of Cermelli e

Arthur Douillard 116 Dec 14, 2022
Implementation of CVPR 2020 Dual Super-Resolution Learning for Semantic Segmentation

Dual super-resolution learning for semantic segmentation 2021-01-02 Subpixel Update Happy new year! The 2020-12-29 update of SISR with subpixel conv p

Sam 79 Nov 24, 2022
Public repository of the 3DV 2021 paper "Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds"

Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds Björn Michele1), Alexandre Boulch1), Gilles Puy1), Maxime Bucher1) and Rena

valeo.ai 15 Dec 22, 2022
code for paper"A High-precision Semantic Segmentation Method Combining Adversarial Learning and Attention Mechanism"

PyTorch implementation of UAGAN(U-net Attention Generative Adversarial Networks) This repository contains the source code for the paper "A High-precis

Tong 8 Apr 25, 2022
Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation (CVPR 2022)

CCAM (Unsupervised) Code repository for our paper "CCAM: Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localizati

Computer Vision Insitute, SZU 113 Dec 27, 2022