Pixel-wise segmentation on VOC2012 dataset using pytorch.

Overview

PiWiSe

Pixel-wise segmentation on the VOC2012 dataset using pytorch.

For a more complete implementation of segmentation networks checkout semseg.

Note:

  • FCN differs from original implementation see this issue
  • SegNet does not match original paper performance see here
  • PSPNet misses "atrous convolution" (conv layers of ResNet101 should be amended to preserve image size)

Keeping this in mind feel free to PR. Thank you!

Setup

See dataset examples here.

Download

Download image archive and extract and do:

mkdir data
mv VOCdevkit/VOC2012/JPEGImages data/images
mv VOCdevkit/VOC2012/SegmentationClass data/classes
rm -rf VOCdevkit

Install

We recommend using pyenv:

pyenv virtualenv 3.6.0 piwise
pyenv activate piwise

then install requirements with pip install -r requirements.txt.

Usage

For latest documentation use:

python main.py --help

Supported model parameters are fcn8, fcn16, fcn32, unet, segnet1, segnet2, pspnet.

Training

If you want to have visualization open an extra tab with:

python -m visdom.server -port 5000

Train the SegNet model 30 epochs with cuda support, visualization and checkpoints every 100 steps:

python main.py --cuda --model segnet2 train --datadir data \
    --num-epochs 30 --num-workers 4 --batch-size 4 \
    --steps-plot 50 --steps-save 100

Evaluation

Then we want to do semantic segmentation on foo.jpg:

python main.py --model segnet2 --state segnet2-30-0 eval foo.jpg foo.png

The segmented class image can now be found at foo.png.

Results

These are some results based on segnet after 40 epoches. Set

loss_weights[0] = 1 / 1

to deal gracefully with the unbalanced problem.

Input Output Ground Truth
Comments
  • Semantic segmentation for binary masks (2 class: background/foreground object)

    Semantic segmentation for binary masks (2 class: background/foreground object)

    Hi @bodokaiser Do you know how to adapt this code for binary masks? (background = black, foreground = white) I changed the number of classes to 2: #NUM_CLASSES = 22 NUM_CLASSES = 2 But some erros still occurs:

    Traceback (most recent call last):
      File "main2.py", line 164, in <module>
        main(parser.parse_args())
      File "main2.py", line 139, in main
        train(args, model)
      File "main2.py", line 74, in train
        loss = criterion(outputs, targets[:, 0])
      File "/home/ubuntu/src/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 224, in __call__
        result = self.forward(*input, **kwargs)
      File "/home/ubuntu/pytorch/piwise/piwise/criterion.py", line 13, in forward
        return self.loss(F.log_softmax(outputs), targets)
      File "/home/ubuntu/src/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 224, in __call__
        result = self.forward(*input, **kwargs)
      File "/home/ubuntu/src/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 132, in forward
        self.ignore_index)
      File "/home/ubuntu/src/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/functional.py", line 674, in nll_loss
        return _functions.thnn.NLLLoss2d.apply(input, target, weight, size_average, ignore_index)
      File "/home/ubuntu/src/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/_functions/thnn/auto.py", line 47, in forward
        output, *ctx.additional_args)
    RuntimeError: weight tensor should be defined either for all or no classes at /pytorch/torch/lib/THCUNN/generic/SpatialClassNLLCriterion.cu:28
    (pytorch) ubuntu@ip-172-31-85-122:~/pytorch/piwise$
    
    opened by andrewssobral 9
  • Evaluation error

    Evaluation error

    In the assessment there was a mistake, I was not familiar with image segmentation, the problem is here RuntimeError: inconsistent tensor size, expected tensor [256 x 256] and mask [256] to have the same number of elements, but got 65536 and 256 elements respectively at d:\projects\pytorch\torch\lib\th\generic/THTensorMath.c:138 I feel that there is a problem here for label in range(1, len(self.cmap)): mask = gray_image[0] == label color_image[0][mask] = self.cmap[label][0] color_image[1][mask] = self.cmap[label][1] color_image[2][mask] = self.cmap[label][2]

    opened by leemathew1998 4
  • Does the relization of the Segnet in network.py match the paper of segnet?

    Does the relization of the Segnet in network.py match the paper of segnet?

    Hello I read the the relization of the Segnet in network.py and I have a question. In the paper of Segnet,the last layer boxed in the picture below is (one Upsampling + one Conv +one Conv) default

    but in the relization of the Segnet in network.py (https://github.com/bodokaiser/piwise/blob/50f8be658693dd27bfa463cb00d1acec7e869fc4/piwise/network.py#L275),the layer metioned above is (one Upsampling + one Conv) ,it lose one Conv layer. default

    Thank you

    opened by mlx123 2
  • Syntax Error in main.py

    Syntax Error in main.py

    Throws a Syntax Error when running the main.py file

    File "main.py", line 84
        f'input (epoch: {epoch}, step: {step})')
                                              ^
    SyntaxError: invalid syntax
    
    opened by sangeet259 2
  • SyntaxError: invalid syntax in main.py

    SyntaxError: invalid syntax in main.py

    New to pytorch, after data download, I occured this issue:

    root@2d5d0934e049:/home/yuanshuai/code/piwise# python main.py 
      File "main.py", line 84
        f'input (epoch: {epoch}, step: {step})')
                                              ^
    SyntaxError: invalid syntax
    
    opened by ysh329 2
  • Probability maps instead of Binary maps

    Probability maps instead of Binary maps

    Hello, In the evaluation phase, your code outpus binary segmentation maps (for a binary segmentation task), while there cases in which a the real-value probability map is desired. How can I change your code to get the probability map?

    Thanks Saeed

    opened by saeedizadi 2
  • update models

    update models

    opened by bodokaiser 1
  • It's different with the standard SegNet

    It's different with the standard SegNet

    I think the most contribution in segnet is the idx-maxpooling. you can use the F.max_unpool2d(idx, x) to replace the nn.upsample(). The original upsample can't fit the loss of location information.

    opened by deeptoby 9
  • About an inexplicable bug

    About an inexplicable bug

    I ran this program on my own computer very well, but when I ran it on another computer, such a error happened: RuntimeError: cuda runtime error(59): device-side assert triggered when running transfer_learning_tutorial This error seems to be caused by a problem with the labels. Can you tell me how to solve this bug?

    opened by zhulanyun 1
  • This question is about how to eliminate the white box in the image

    This question is about how to eliminate the white box in the image

    Every label of image have a white margin , I want to ignore the margin when I train net ,But I do not know how to do,So i ask you this question Could you help me?

    opened by mshmoon 1
  • A question about the size of output imgae

    A question about the size of output imgae

    The code need input size (256 ,256) ,and output size is (256,256).This limit the ability of the practice.So I want to extend the net ability to arbitrary input size .If I do ,the code need adjustment? I can not understand the code in depth. Please help def colormap(n): cmap=np.zeros([n, 3]).astype(np.uint8)

    for i in np.arange(n):
        r, g, b = np.zeros(3)
    
        for j in np.arange(8):
            r = r + (1<<(7-j))*((i&(1<<(3*j))) >> (3*j))
            g = g + (1<<(7-j))*((i&(1<<(3*j+1))) >> (3*j+1))
            b = b + (1<<(7-j))*((i&(1<<(3*j+2))) >> (3*j+2))
    
        cmap[i,:] = np.array([r, g, b])
    
    return cmap
    
    opened by mshmoon 1
Owner
Bodo Kaiser
Currently working on quantum optical communication and a software-as-a-service product for company-startup cooperations.
Bodo Kaiser
Retinal Vessel Segmentation with Pixel-wise Adaptive Filters (ISBI 2022)

Retinal Vessel Segmentation with Pixel-wise Adaptive Filters (ISBI 2022) Introdu

anonymous 14 Oct 27, 2022
Official code of Retinal Vessel Segmentation with Pixel-wise Adaptive Filters and Consistency Training

Official code of Retinal Vessel Segmentation with Pixel-wise Adaptive Filters and Consistency Training (ISBI 2022)

anonymous 7 Feb 10, 2022
This is an official implementation of "Polarized Self-Attention: Towards High-quality Pixel-wise Regression"

Polarized Self-Attention: Towards High-quality Pixel-wise Regression This is an official implementation of: Huajun Liu, Fuqiang Liu, Xinyi Fan and Don

DeLightCMU 212 Jan 8, 2023
Code for "PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation" CVPR 2019 oral

Good news! We release a clean version of PVNet: clean-pvnet, including how to train the PVNet on the custom dataset. Use PVNet with a detector. The tr

ZJU3DV 722 Dec 27, 2022
Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera.

Tools to create pixel-wise object masks, bounding box labels (2D and 3D) and 3D object model (PLY triangle mesh) for object sequences filmed with an RGB-D camera. This project prepares training and testing data for various deep learning projects such as 6D object pose estimation projects singleshotpose, as well as object detection and instance segmentation projects.

null 305 Dec 16, 2022
HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation Official PyTorch Implementation

: We present a novel, real-time, semantic segmentation network in which the encoder both encodes and generates the parameters (weights) of the decoder. Furthermore, to allow maximal adaptivity, the weights at each decoder block vary spatially. For this purpose, we design a new type of hypernetwork, composed of a nested U-Net for drawing higher level context features

Yuval Nirkin 182 Dec 14, 2022
Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Pixel-Level Cycle Association This is the Pytorch implementation of our NeurIPS 2020 Oral paper Pixel-Level Cycle Association: A New Perspective for D

null 87 Oct 19, 2022
Synthetic LiDAR sequential point cloud dataset with point-wise annotations

SynLiDAR dataset: Learning From Synthetic LiDAR Sequential Point Cloud This is official repository of the SynLiDAR dataset. For technical details, ple

null 78 Dec 27, 2022
Exploring Cross-Image Pixel Contrast for Semantic Segmentation

Exploring Cross-Image Pixel Contrast for Semantic Segmentation Exploring Cross-Image Pixel Contrast for Semantic Segmentation, Wenguan Wang, Tianfei Z

Tianfei Zhou 510 Jan 2, 2023
Pixel Consensus Voting for Panoptic Segmentation (CVPR 2020)

Implementation for Pixel Consensus Voting (CVPR 2020). This codebase contains the essential ingredients of PCV, including various spatial discretizati

Haochen 23 Oct 25, 2022
Per-Pixel Classification is Not All You Need for Semantic Segmentation

MaskFormer: Per-Pixel Classification is Not All You Need for Semantic Segmentation Bowen Cheng, Alexander G. Schwing, Alexander Kirillov [arXiv] [Proj

Facebook Research 1k Jan 8, 2023
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Jiwoon Ahn 337 Dec 15, 2022
[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

Ye Du 96 Dec 30, 2022
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (NeurIPS2021 Benchmark and Dataset Track)

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

Kingdrone 174 Dec 22, 2022
Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data based on Pytorch Framework

VFedPCA+VFedAKPCA This is the official source code for the Paper: Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-

John 9 Sep 18, 2022
📦 PyTorch based visualization package for generating layer-wise explanations for CNNs.

Explainable CNNs ?? Flexible visualization package for generating layer-wise explanations for CNNs. It is a common notion that a Deep Learning model i

Ashutosh Hathidara 183 Dec 15, 2022
Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Transformer in Transformer Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image c

Phil Wang 272 Dec 23, 2022
A unofficial pytorch implementation of PAN(PSENet2): Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network

Efficient and Accurate Arbitrary-Shaped Text Detection with Pixel Aggregation Network Requirements pytorch 1.1+ torchvision 0.3+ pyclipper opencv3 gcc

zhoujun 400 Dec 26, 2022
This is code to fit per-pixel environment map with spherical Gaussian lobes, using LBFGS optimization

Spherical Gaussian Optimization This is code to fit per-pixel environment map with spherical Gaussian lobes, using LBFGS optimization. This code has b

null 41 Dec 14, 2022