PyTorch implementation of PSPNet

Overview

PSPNet with PyTorch

Unofficial implementation of "Pyramid Scene Parsing Network" (https://arxiv.org/abs/1612.01105). This repository is just for caffe to pytorch model conversion and evaluation.

Requirements

  • pytorch
  • click
  • addict
  • pydensecrf
  • protobuf

Preparation

Instead of building the author's caffe implementation, you can convert off-the-shelf caffemodels to pytorch models via the caffe.proto.

1. Compile the caffe.proto for Python API

This step can be skipped. FYI.
Download the author's caffe.proto into the libs, not the one in the original caffe.

# For protoc command
pip install protobuf
# This generates ./caffe_pb2.py
protoc --python_out=. caffe.proto

2. Model conversion

  1. Find the caffemodels on the author's page (e.g. pspnet50_ADE20K.caffemodel) and store them to the data/models/ directory.
  2. Convert the caffemodels to .pth file.
python convert.py -c <PATH TO YAML>

Demo

python demo.py -c <PATH TO YAML> -i <PATH TO IMAGE>
  • With a --no-cuda option, this runs on CPU.
  • With a --crf option, you can perform a CRF postprocessing.

demo

Evaluation

PASCAL VOC2012 only. Please set the dataset path in config/voc12.yaml.

python eval.py -c config/voc12.yaml

88.1% mIoU (SS) and 88.6% mIoU (MS) on validation set.
NOTE: 3 points lower than caffe implementation. WIP

  • SS: averaged prediction with flipping (2x)
  • MS: averaged prediction with multi-scaling (6x) and flipping (2x)
  • Both: No CRF post-processing

References

You might also like...
Generic template to bootstrap your PyTorch project with PyTorch Lightning, Hydra, W&B, and DVC.

NN Template Generic template to bootstrap your PyTorch project. Click on Use this Template and avoid writing boilerplate code for: PyTorch Lightning,

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

This repository holds NVIDIA-maintained utilities to streamline mixed precision and distributed training in Pytorch. Some of the code here will be included in upstream Pytorch eventually. The intention of Apex is to make up-to-date utilities available to users as quickly as possible.

Objective of the repository is to learn and build machine learning models using Pytorch. 30DaysofML Using Pytorch
Objective of the repository is to learn and build machine learning models using Pytorch. 30DaysofML Using Pytorch

30 Days Of Machine Learning Using Pytorch Objective of the repository is to learn and build machine learning models using Pytorch. List of Algorithms

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch
Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks
Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks

Amazon Forest Computer Vision Satellite Image tagging code using PyTorch / Keras Here is a sample of images we had to work with Source: https://www.ka

The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch.
The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch.

This is a curated list of tutorials, projects, libraries, videos, papers, books and anything related to the incredible PyTorch. Feel free to make a pu

Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks
Amazon Forest Computer Vision: Satellite Image tagging code using PyTorch / Keras with lots of PyTorch tricks

Amazon Forest Computer Vision Satellite Image tagging code using PyTorch / Keras Here is a sample of images we had to work with Source: https://www.ka

A bunch of random PyTorch models using PyTorch's C++ frontend
A bunch of random PyTorch models using PyTorch's C++ frontend

PyTorch Deep Learning Models using the C++ frontend Gettting started Clone the repo 1. https://github.com/mrdvince/pytorchcpp 2. cd fashionmnist or

PyTorch Autoencoders - Implementing a Variational Autoencoder (VAE) Series in Pytorch.

PyTorch Autoencoders Implementing a Variational Autoencoder (VAE) Series in Pytorch. Inspired by this repository Model List check model paper conferen

Comments
  • What is yaml?

    What is yaml?

    I have a question about the yaml. What is yaml' here? In caffe, models are save in .caffemodel, non yaml. I downloaded thepspnet101_VOC2012.caffemodelfrom @hszhao's repo and put it indata/modelsthen I ranpython convert.py -c data/models/pspnet101_VOC2012.caffemodel` but it generated following error:

    Traceback (most recent call last):
      File "convert.py", line 161, in <module>
        main()
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
        return self.main(*args, **kwargs)
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/click/core.py", line 697, in main
        rv = self.invoke(ctx)
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
        return callback(*args, **kwargs)
      File "convert.py", line 117, in main
        CONFIG = Dict(yaml.load(open(config)))
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/yaml/__init__.py", line 69, in load
        loader = Loader(stream)
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/yaml/loader.py", line 34, in __init__
        Reader.__init__(self, stream)
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/yaml/reader.py", line 85, in __init__
        self.determine_encoding()
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/yaml/reader.py", line 135, in determine_encoding
        self.update(1)
      File "/mnt/hdd1/cao/.global_env/local/lib/python2.7/site-packages/yaml/reader.py", line 165, in update
        exc.encoding, exc.reason)
    yaml.reader.ReaderError: 'utf8' codec can't decode byte #xa2: invalid start byte
      in "data/models/pspnet101_VOC2012.caffemodel", position 2
    
    
    opened by cao-nv 2
  • Variable sized image?

    Variable sized image?

    I wonder whether a pretrained PSPnet is able to handle variable size input. I am using the ade20k model, and I noticed that the input image to the model is by default 473x473. And if I changed it to 512x512, then the segmentation mask would be somewhat off by a few pixels.

    I wonder whether I should still input the image as size 473x473, but later using bilinear upsampling to enlarge the mask to 512x512?

    opened by linzhiqiu 1
  • Error in convert caffe model to pytorch version

    Error in convert caffe model to pytorch version

    Hi, thanks for your nice work! I'm using your code but get some error in model transform, these are my error info:


    conv5_3_pool6_conv/bn -> ppm.stages.0.conv.bn.weight : Copied conv5_3_pool6_conv/bn -> ppm.stages.0.conv.bn.running_var : Copied conv5_3_pool6_conv/bn -> ppm.stages.0.conv.bn.bias : Copied conv5_3_pool6_conv/bn -> ppm.stages.0.conv.bn.running_mean : Copied conv5_4 -> final.conv5_4.conv.weight : Copied conv5_4/bn -> final.conv5_4.bn.weight : Copied conv5_4/bn -> final.conv5_4.bn.running_var : Copied conv5_4/bn -> final.conv5_4.bn.bias : Copied conv5_4/bn -> final.conv5_4.bn.running_mean : Copied conv6 -> final.conv6.weight : Copied conv6 -> final.conv6.bias : Copied conv4_24 -> aux.conv4_aux.conv.weight : Copied conv4_24/bn -> aux.conv4_aux.bn.weight : Copied conv4_24/bn -> aux.conv4_aux.bn.running_var : Copied conv4_24/bn -> aux.conv4_aux.bn.bias : Copied conv4_24/bn -> aux.conv4_aux.bn.running_mean : Copied Traceback (most recent call last): File "convert.py", line 168, in main() File "/home/wzj/opt/miniconda2/envs/seg/lib/python2.7/site-packages/click/core.py", line 722, in call return self.main(*args, **kwargs) File "/home/wzj/opt/miniconda2/envs/seg/lib/python2.7/site-packages/click/core.py", line 697, in main rv = self.invoke(ctx) File "/home/wzj/opt/miniconda2/envs/seg/lib/python2.7/site-packages/click/core.py", line 895, in invoke return ctx.invoke(self.callback, **ctx.params) File "/home/wzj/opt/miniconda2/envs/seg/lib/python2.7/site-packages/click/core.py", line 535, in invoke return callback(*args, **kwargs) File "convert.py", line 157, in main values = values.view_as(own_state[param_name]) File "/home/wzj/opt/miniconda2/envs/seg/lib/python2.7/site-packages/torch/tensor.py", line 198, in view_as return self.view(tensor.size()) RuntimeError: invalid argument 2: size '[21 x 256 x 3 x 3]' is invalid for input with 5376 elements at /pytorch/torch/lib/TH/THStorage.c:41


    I'm using python3.5 and pytorch0.3.0

    Could you please help me fix this, or just share the model file you have transformed, thanks alot!

    opened by zhijiew 1
  • accuracy is much lower

    accuracy is much lower

    when I convert the voc101 model to pytorch version , I test on VOC2012 val.txt, but the mean IoU is 79.6% , much lower than the author given which is 85.41%, is there any other details I ignored

    opened by luhc15 8
Owner
Kazuto Nakashima
Kazuto Nakashima
ALBERT-pytorch-implementation - ALBERT pytorch implementation

ALBERT-pytorch-implementation developing... 모델의 개념이해를 돕기 위한 구현물로 현재 변수명을 상세히 적었고

BG Kim 3 Oct 6, 2022
An essential implementation of BYOL in PyTorch + PyTorch Lightning

Essential BYOL A simple and complete implementation of Bootstrap your own latent: A new approach to self-supervised Learning in PyTorch + PyTorch Ligh

Enrico Fini 48 Sep 27, 2022
RealFormer-Pytorch Implementation of RealFormer using pytorch

RealFormer-Pytorch Implementation of RealFormer using pytorch. Includes comparison with classical Transformer on image classification task (ViT) wrt C

Simo Ryu 90 Dec 8, 2022
A PyTorch implementation of the paper Mixup: Beyond Empirical Risk Minimization in PyTorch

Mixup: Beyond Empirical Risk Minimization in PyTorch This is an unofficial PyTorch implementation of mixup: Beyond Empirical Risk Minimization. The co

Harry Yang 121 Dec 17, 2022
A pytorch implementation of Pytorch-Sketch-RNN

Pytorch-Sketch-RNN A pytorch implementation of https://arxiv.org/abs/1704.03477 In order to draw other things than cats, you will find more drawing da

Alexis David Jacq 172 Dec 12, 2022
PyTorch implementation of Advantage async actor-critic Algorithms (A3C) in PyTorch

Advantage async actor-critic Algorithms (A3C) in PyTorch @inproceedings{mnih2016asynchronous, title={Asynchronous methods for deep reinforcement lea

LEI TAI 111 Dec 8, 2022
Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

PyTorch implementation of 'Denoising Diffusion Probabilistic Models' This reposi

Arthur Juliani 76 Jan 7, 2023
Fang Zhonghao 13 Nov 19, 2022
RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

RETRO - Pytorch (wip) Implementation of RETRO, Deepmind's Retrieval based Attent

Phil Wang 556 Jan 4, 2023
HashNeRF-pytorch - Pure PyTorch Implementation of NVIDIA paper on Instant Training of Neural Graphics primitives

HashNeRF-pytorch Instant-NGP recently introduced a Multi-resolution Hash Encodin

Yash Sanjay Bhalgat 616 Jan 6, 2023