code and models for "Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation"

Last update: Jun 18, 2022

Related tags

Deep Learning LRR

Overview

Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation

This repository contains code and models for the method described in:

Golnaz Ghiasi, Charless C. Fowlkes, "Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation", ECCV 2016

The code is written in Matlab, it uses Matconvnet library and is based on the following repository:

matconvnet-fcn

This code is tested on Linux using matconvnet v1.0-beta20 and cuDNN 5

Testing pre-trained models

Download pre-trained models and extract it into models directory.

Testing pre-trained model on PASCAL VOC validation data

Specify matconvnet path in "LRRTestOnPascal.m" and execute it.

Testing pre-trained model on Cityscape validation data

Download "gtFine_trainvaltest.zip" and "leftImg8bit_trainvaltest.zip" from Cityscapes dataset website, unzip them. Specify their path ("opts.dataDir") and matconvnet path ("path_to_matconvnet") in "LRRTestOnCityScape.m" and execute it.

Training LRR on PASCAL VOC training data

Specify matconvnet path in "LRR4xTrainVGG16Pascal.m" and execute it.

Issues, Questions, etc

Please contact "gghiasi @ ics.uci.edu"

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Comments

Error in LRRAddMasking

Hi, I'm trying to run the demo LRR4XTrainVGG16Pascal.m code on the Pascal dataset. The code runs fine up until the point that it tries to train a 16x submodel. Then I get the following error:

training submodel: 16x Undefined variable "dagnn" or class "dagnn.Neg".

Error in LRRAddMasking (line 21) net.addLayer(['neg_prob_' pre_up_name], dagnn.Neg(), ['prob_' pre_up_name], ['neg_prob_' pre_up_name], {});

Error in LRR4xTrainVGG16Pascal (line 254) net = LRRAddMasking(net, upsample_fac(step), upsample_fac(step)/opts.rec_upsample, upsample_2x_pre_layer);

I've looked and the dagnn variable is undefined. Is this a known bug, or is it potentially linked to my MatConvNet setup (I have the latest version of that).

I'm running the code on a TitanX GPU. Any help is appreciated.

opened by JoelChev 5
Can not testing pre-trained model on Cityscape
Hi, i am trying to test the pretrained model on Cityscapes, but get no output.

I downloaded the gtFine_trainvaltest.zip" and "leftImg8bit_trainvaltest.zip', unziped it and saved it in /data/CityScapes/. So the paths are: /data/CityScapes/gtFine/val/frankfurt/... /data/CityScapes/leftImg8bit/val/frankfurt/...

But when i execute, it says: Number of validation data: 0

The output of imdb is the following:

`root_path: 'data/CityScapes/' anno_path: 'data/CityScapes//%s/%s/%s/%s' img_path: 'data/CityScapes//leftImg8bit/%s/%s/%s_leftImg8bit.png' sets: [1x1 struct] classes: [1x1 struct] images: [1x1 struct] num_classes: 19 classes_names: {1x19 cell}`

Maybe the two "//" in the path cause the problem?
opened by Timo-hab 1
I got an ERROR about the dimensions of the input when I use the model to test on the Cityscapes dataset.

Warning: DotProduct layer: the dimensions of the input variables is not the same.

In DotProduct>DotProduct.getOutputSizes at 33 In DagNN.getVarSizes at 38 In DagNN.print at 70 In model2dot at 58 In LRRTestOnCityScape at 48 Warning: Sum layer: the dimensions of the input variables is not the same. In Sum>Sum.getOutputSizes at 31 In DagNN.getVarSizes at 38 In DagNN.print at 70 In model2dot at 58 In LRRTestOnCityScape at 48 Warning: DotProduct layer: the dimensions of the input variables is not the same. In DotProduct>DotProduct.getOutputSizes at 33 In DagNN.getVarSizes at 38 In DagNN.print at 70 In model2dot at 58 In LRRTestOnCityScape at 48 Warning: Sum layer: the dimensions of the input variables is not the same. In Sum>Sum.getOutputSizes at 31 In DagNN.getVarSizes at 38 In DagNN.print at 70 In model2dot at 58 In LRRTestOnCityScape at 48 Warning: DotProduct layer: the dimensions of the input variables is not the same. In DotProduct>DotProduct.getOutputSizes at 33 In DagNN.getVarSizes at 38 In DagNN.print at 70 In model2dot at 58 In LRRTestOnCityScape at 48 Warning: Sum layer: the dimensions of the input variables is not the same. In Sum>Sum.getOutputSizes at 31 In DagNN.getVarSizes at 38 In DagNN.print at 70 In model2dot at 58 In LRRTestOnCityScape at 48 Model exported to models/LRR4x-VGG16-CityScapes-coarse-and-fine/model-vis.dot. visualization of the model saved to models/LRR4x-VGG16-CityScapes-coarse-and-fine/model-vis.png [Please exit and restart MATLAB]>> [y or n]>>n

opened by wangq95 0
error on fine tuning LRR with multi-GPU mode

I try to fine tune LRR on my dataset an I set the training to the multi-gpu mode by setting : opts.train.gpus = [1:3] same as fcnTrain.m function which works on multi-gpu mode in this way. But i got error while training process want to use "getBatch" function. How can i resolve this error and use multi-gpu mode?

opened by Mirsadeghi 0

LRRTestOnPascal() doesn't work

Hi, i was trying to run the demo, i got this error:

Attempt to execute SCRIPT vl_nnconv as a function

Error in dagnn.Conv/forward (line 11)
      outputs{1} = vl_nnconv(...

Error in dagnn.Layer/forwardAdvanced (line 85)
      outputs = obj.forward(inputs, {net.params(par).value}) ;

Error in dagnn.DagNN/eval (line 91)
  obj.layers(l).block.forwardAdvanced(obj.layers(l)) ;

Error in LRRTestOnPascal (line 143)
        net.eval({inputVar, net_input});

any ideas? thanks.

opened by yortuc 0

Converting LRR cityscapes to caffe model

Hello!

The LRR model trained on cityscapes works really well on my data and I would like to use it and deploy it as a ros node for use in a robot.

To do this I would prefer to have the trained LRR as a caffemodel. Do you know if it is possible to convert the LRR network to a caffemodel?

opened by steve3nto 0
training demo doesn't work

When I run the training demo LRR4xTrainVGG16Pascal, it goes fine at the beginning. However, after several iterations, I got the following output:

obj_dil_seg32x: NaN obj_ero_seg32x: NaN objective_32x: NaN

What's wrong with it? I didn't change anything expect for the directories of the dataset.

opened by cslxiao 2

code and models for "Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation"

Related tags

Overview

Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation

Testing pre-trained models

Testing pre-trained model on PASCAL VOC validation data

Testing pre-trained model on Cityscape validation data

Training LRR on PASCAL VOC training data

Issues, Questions, etc

Comments

Error in LRRAddMasking

Can not testing pre-trained model on Cityscape

I got an ERROR about the dimensions of the input when I use the model to test on the Cityscapes dataset.

error on fine tuning LRR with multi-GPU mode

LRRTestOnPascal() doesn't work

Converting LRR cityscapes to caffe model

training demo doesn't work

Owner

This repository contains the code and models necessary to replicate the results of paper: How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective

This repository contains the code and models necessary to replicate the results of paper: How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective

Code for pre-training CharacterBERT models (as well as BERT models).

pyhsmm - library for approximate unsupervised inference in Bayesian Hidden Markov Models (HMMs) and explicit-duration Hidden semi-Markov Models (HSMMs), focusing on the Bayesian Nonparametric extensions, the HDP-HMM and HDP-HSMM, mostly with weak-limit approximations.

This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

Reference implementation of code generation projects from Facebook AI Research. General toolkit to apply machine learning to code, from dataset creation to model training and evaluation. Comes with pretrained models.

OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale

PyTorch implementation and pretrained models for XCiT models. See XCiT: Cross-Covariance Image Transformer

This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)

Quickly comparing your image classification models with the state-of-the-art models (such as DenseNet, ResNet, ...)

Pre-trained BERT Models for Ancient and Medieval Greek, and associated code for LaTeCH 2021 paper titled - "A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek"

Source code, datasets and trained models for the paper Learning Advanced Mathematical Computations from Examples (ICLR 2021), by François Charton, Amaury Hayat (ENPC-Rutgers) and Guillaume Lample

Source code and dataset for ACL2021 paper: "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning".

a delightful machine learning tool that allows you to train, test and use models without writing code

Code, Models and Datasets for OpenViDial Dataset

Ludwig is a toolbox that allows to train and evaluate deep learning models without the need to write code.

Ludwig is a toolbox that allows to train and evaluate deep learning models without the need to write code.