Semantic segmentation task for ADE20k & cityscapse dataset, based on several models.

Last update: Oct 13, 2022

Overview

semantic-segmentation-tensorflow

This is a Tensorflow implementation of semantic segmentation models on MIT ADE20K scene parsing dataset and Cityscapes dataset We re-produce the inference phase of several models, including PSPNet, FCN, and ICNet by transforming the released pre-trained weights into tensorflow format, and apply on handcraft models. Also, we refer to ENet from freg856 github. Still working on task integrated.

Models

PSPNet
FCN
ENet
ICNet

...to be continue

Install

Get corresponding transformed pre-trained weights, and put into model directory:

FCN	PSPNet	ICNet
Google drive	Google drive	Google drive

Inference

Run following command:

python inference.py --img-path /Path/To/Image --dataset Model_Type

Arg list

--model - choose from "icnet"/"pspnet"/"fcn"/"enet"

Import module in your code:

from model import FCN8s, PSPNet50, ICNet, ENet

model = PSPNet50() # or another model

model.read_input(img_path)  # read image data from path

sess = tf.Session(config=config)
init = tf.global_variables_initializer()
sess.run(init)

model.load(model_path, sess)  # load pretrained model
preds = model.forward(sess) # Get prediction

Results

ade20k

Input Image	PSPNet	FCN

cityscapes

Input Image	ICNet	ENet

Citation

@inproceedings{zhao2017pspnet,
  author = {Hengshuang Zhao and
            Jianping Shi and
            Xiaojuan Qi and
            Xiaogang Wang and
            Jiaya Jia},
  title = {Pyramid Scene Parsing Network},
  booktitle = {Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2017}
}

Scene Parsing through ADE20K Dataset. B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso and A. Torralba. Computer Vision and Pattern Recognition (CVPR), 2017. (http://people.csail.mit.edu/bzhou/publication/scene-parse-camera-ready.pdf)

@inproceedings{zhou2017scene,
    title={Scene Parsing through ADE20K Dataset},
    author={Zhou, Bolei and Zhao, Hang and Puig, Xavier and Fidler, Sanja and Barriuso, Adela and Torralba, Antonio},
    booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
    year={2017}
}

Semantic Understanding of Scenes through ADE20K Dataset. B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso and A. Torralba. arXiv:1608.05442. (https://arxiv.org/pdf/1608.05442.pdf)

@article{zhou2016semantic,
  title={Semantic understanding of scenes through the ade20k dataset},
  author={Zhou, Bolei and Zhao, Hang and Puig, Xavier and Fidler, Sanja and Barriuso, Adela and Torralba, Antonio},
  journal={arXiv preprint arXiv:1608.05442},
  year={2016}
}

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

8 Nov 21, 2022

The code uses SegFormer for Semantic Segmentation on Drone Dataset.

SegFormer_Segmentation The code uses SegFormer for Semantic Segmentation on Drone Dataset. The details for the SegFormer can be obtained from the foll

1 May 8, 2022

Official repository for "Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems"

Action-Based Conversations Dataset (ABCD) This respository contains the code and data for ABCD (Chen et al., 2021) Introduction Whereas existing goal-

49 Oct 9, 2022

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP Abstract: We introduce a method that allows to automatically se

134 Dec 19, 2022

TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

147 Dec 3, 2022

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

ADE20k Semantic segmentation with MAE Getting started Install the mmsegmentation

97 Dec 17, 2022

This is the official repo for TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transformations at CVPR'21. According to some product reasons, we are not planning to release the training/testing codes and models. However, we will release the dataset and the scripts to prepare the dataset.

TransFill-Reference-Inpainting This is the official repo for TransFill: Reference-guided Image Inpainting by Merging Multiple Color and Spatial Transf

80 Dec 8, 2022

Multi-task yolov5 with detection and segmentation based on yolov5

YOLOv5DS Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0) decoupled head anchor free segmentation head README中文 Ablation

150 Dec 30, 2022

PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Reference-Based-Sketch-Image-Colorization-ImageNet This is a PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization usin

11 Jul 28, 2022

Comments

Error when using ICNet model

Hi!

First of all, thanks for sharing your very helpful code. Following the README.md, I have downloaded icnet.npy from Google Drive link to the model/cityscapes folder.

However, when I run

python inference.py --img-path input/outdoor_1.png --model icnet

the following error happens:

Traceback (most recent call last):
  File "inference.py", line 65, in <module>
    main()
  File "inference.py", line 44, in main
    model = ICNet()
  File "/home/tadashi/ML/semantic-segmentation-tensorflow/model.py", line 907, in __init__
    super().__init__({'data': self.img_tf}, num_classes, is_training)
TypeError: super() takes at least 1 argument (0 given)

opened by htadashi 2

Cityscapes test data

Can you tell me how to test the mean iou in the cityscape test set online? Because the cityscape test set has no tags, online testing is required, but I don't know what to do.I uploaded the test results as he asked, but there was always Content of zip archive does not match specifications. Not all predictions for all images were found. Thank you very much！

opened by InstantWindy 0
about the training

Hi, @hellochick ,

Thanks for releasing such useful package. However, only inference tools are provided. Would you also release the training part for them?

Thanks~

opened by amiltonwong 1
How to use ENet?

I am trying to use ENet for segmentation, but from the README.md it is not clear how I may do that. I tried passing the model_path as os.path.join('model','cityscapes','enet.ckpt.meta')

But I get the following error:

InvalidArgumentError (see above for traceback): No OpKernel was registered to support Op 'MaxPoolWithArgmax' with these attrs. Registered devices: [CPU], Registered kernels:

[[Node: MaxPoolWithArgmax = MaxPoolWithArgmaxT=DT_FLOAT, Targmax=DT_INT64, ksize=[1, 2, 2, 1], padding="SAME", strides=[1, 2, 2, 1]]]

Note: I fixed your typo at line 898, where you had "variabl" instead of "variable"

opened by srcolinas 0

Semantic segmentation task for ADE20k & cityscapse dataset, based on several models.

Related tags

Overview

semantic-segmentation-tensorflow

Models

...to be continue

Install

Inference

Arg list

Import module in your code:

Results

ade20k

cityscapes

Citation

You might also like...

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

The code uses SegFormer for Semantic Segmentation on Drone Dataset.

Official repository for "Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems"

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Multi-task yolov5 with detection and segmentation based on yolov5

PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Comments

Error when using ICNet model

Cityscapes test data

about the training

How to use ENet?

Owner

HsuanKung Yang

Sematic-Segmantation - Semantic Segmentation on MIT ADE20K dataset in PyTorch

Siamese-nn-semantic-text-similarity - A repository containing comprehensive Neural Networks based PyTorch implementations for the semantic text similarity task

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (NeurIPS2021 Benchmark and Dataset Track)

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.

Tensorflow2 Keras-based Semantic Segmentation Models Implementation

Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Semantic Segmentation of images using PixelLib with help of Pascalvoc dataset trained with Deeplabv3+ framework.