TransferNet: Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

Last update: Jun 29, 2022

Related tags

Deep Learning TransferNet

Overview

TransferNet: Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

Created by Seunghoon Hong, Junhyuk Oh, Honglak Lee and Bohyung Han

Project page: [http://cvlab.postech.ac.kr/research/transfernet/]

Introduction

This repository contains the source code for the semantic segmentation algorithm described in the following paper:

Seunghoon Hong, Junhyuk Oh, Honglak Lee, Bohyung Han, "Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network" In IEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

@inproceedings{HongOLH2016,
  title={Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network},
  author={Hong, Seunghoon and Oh, Junhyuk and Lee, Honglak and Han, Bohyung},
  booktitle={Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on},
  year={2016}
}

Pleae refer to our arXiv tech report for details.

Installation

You need to compile the modified Caffe library in this repository. Please consult Caffe installation guide for details. After installing rquired libraries for Caffe, you need to compile both Caffe and its Matlab interface as follows:

cd caffe
make all
make matcaffe

After installing Caffe, you can download datasets, pre-trained models, and other libraries by following script:

setup.sh

Training

Training procedures are composed of two steps, which are implemented in different directories:

training/1_train_attention : pre-train attention and classification network with image-level class labels.
training/2_train_segmentation : train entire network including a decoder with pixel-wise class labels.

You can run training with following scripts

cd training
./1_train_attention.sh
./2_train_segmentation.sh

Inference

You can run inference on PASCAL VOC 2012 validatoin images using the trained model as follow:

cd inference
matlab -nodesktop -r run_inference

By default, this script will perform an inference on PASCAL VOC 2012 validation images using the pre-trained model. You may need to modify the code if you want to apply the model to different dataset or use the different models.

Licence

This software is for research purpose only. Check LICENSE file for details.

You might also like...

Code repo for "RBSRICNN: Raw Burst Super-Resolution through Iterative Convolutional Neural Network" (Machine Learning and the Physical Sciences workshop in NeurIPS 2021).

RBSRICNN: Raw Burst Super-Resolution through Iterative Convolutional Neural Network An official PyTorch implementation of the RBSRICNN network as desc

6 Nov 14, 2022

Code for "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation".

FPS-Net Code for "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation", accepted by ISPRS journal of Photogrammetry

15 Nov 30, 2022

[IJCAI-2021] A benchmark of data-free knowledge distillation from paper "Contrastive Model Inversion for Data-Free Knowledge Distillation"

DataFree A benchmark of data-free knowledge distillation from paper "Contrastive Model Inversion for Data-Free Knowledge Distillation" Authors: Gongfa

47 Jan 9, 2023

TF2 implementation of knowledge distillation using the "function matching" hypothesis from the paper Knowledge distillation: A good teacher is patient and consistent by Beyer et al.

FunMatch-Distillation TF2 implementation of knowledge distillation using the "function matching" hypothesis from the paper Knowledge distillation: A g

67 Dec 20, 2022

A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

A PyTorch implementation of V-Net Vnet is a PyTorch implementation of the paper V-Net: Fully Convolutional Neural Networks for Volumetric Medical Imag

606 Dec 21, 2022

Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Siamese Deep Neural Networks for Semantic Text Similarity PyTorch A repository c

32 Dec 15, 2022

Build upon neural radiance fields to create a scene-specific implicit 3D semantic representation, Semantic-NeRF

Comments

Custom classificationet.caffemodel

Hello,

I trained TransferNet on my dataset with novel categories. During the inference in Matlab there is classificationet. prototxt and classificationnet.caffemodel adopted. As these files differs from the attentionnet.prototxt and attentionnet.caffemodel, it is imposible to infer on my dataset. How should I modify my trained attentionnet.caffemodel to classificationet.caffemodel ?

Sincerely

Michal

opened by michalvavrecka 0
deploy.prototxt did not pass thru Netscope

Hello,

I am testing Transfernet to modify it to my data. I still face troubles to understand attention module as it is described differently in your equations in paper, your diagram (same paper) and in deploy.prototxt . I tried to visualize the network deploy.prototxt in Netscope - http://ethereon.github.io/netscope/#/editor but it gives me errors. Can you describe me where is the soft_att blob located as there are two contradictory definitions in prototxt, namely:

layer { name: "soft-att" type: "Scale" bottom: "conv5_3" scale_param { scale_factor: 0.004 } # no scaling top: "conv5_3_scaled" }

and

layer { name: "soft-att" type: "LinearSum" bottom: "conv5_3_scaled" bottom: "att" top: "soft-att" }

The Netscope refuse to visualize it (also attention.prototxt) and I am not able to figure out, what is the exact structure of attention module.

Thank you for clarification

Michal

opened by michalvavrecka 0

TransferNet: Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

Related tags

Overview

TransferNet: Learning Transferrable Knowledge for Semantic Segmentation with Deep Convolutional Neural Network

Introduction

Installation

Training

Inference

Licence

You might also like...

Code repo for "RBSRICNN: Raw Burst Super-Resolution through Iterative Convolutional Neural Network" (Machine Learning and the Physical Sciences workshop in NeurIPS 2021).

Code for "FPS-Net: A convolutional fusion network for large-scale LiDAR point cloud segmentation".

[IJCAI-2021] A benchmark of data-free knowledge distillation from paper "Contrastive Model Inversion for Data-Free Knowledge Distillation"

TF2 implementation of knowledge distillation using the "function matching" hypothesis from the paper Knowledge distillation: A good teacher is patient and consistent by Beyer et al.

A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

Build upon neural radiance fields to create a scene-specific implicit 3D semantic representation, Semantic-NeRF

Convolutional 2D Knowledge Graph Embeddings resources

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

Comments

Custom classificationet.caffemodel

deploy.prototxt did not pass thru Netscope

Owner

Train neural network for semantic segmentation (deep lab V3) with pytorch in less then 50 lines of code

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

An implementation of the research paper "Retina Blood Vessel Segmentation Using A U-Net Based Convolutional Neural Network"

Knowledge Distillation Toolbox for Semantic Segmentation

Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

PyTorch implementation of ShapeConv: Shape-aware Convolutional Layer for RGB-D Indoor Semantic Segmentation.

Deep learning (neural network) based remote photoplethysmography: how to extract pulse signal from video using deep learning tools

Scripts for training an AI to play the endless runner Subway Surfers using a supervised machine learning approach by imitation and a convolutional neural network (CNN) for image classification