This project aims at providing a concise, easy-to-use, modifiable reference implementation for semantic segmentation models using PyTorch.

Overview

Semantic Segmentation on PyTorch

English | 简体中文

python-image pytorch-image lic-image

This project aims at providing a concise, easy-to-use, modifiable reference implementation for semantic segmentation models using PyTorch.

Installation

# semantic-segmentation-pytorch dependencies
pip install ninja tqdm

# follow PyTorch installation in https://pytorch.org/get-started/locally/
conda install pytorch torchvision -c pytorch

# install PyTorch Segmentation
git clone https://github.com/Tramac/awesome-semantic-segmentation-pytorch.git

Usage

Train


  • Single GPU training
# for example, train fcn32_vgg16_pascal_voc:
python train.py --model fcn32s --backbone vgg16 --dataset pascal_voc --lr 0.0001 --epochs 50
  • Multi-GPU training
# for example, train fcn32_vgg16_pascal_voc with 4 GPUs:
export NGPUS=4
python -m torch.distributed.launch --nproc_per_node=$NGPUS train.py --model fcn32s --backbone vgg16 --dataset pascal_voc --lr 0.0001 --epochs 50

Evaluation


  • Single GPU evaluating
# for example, evaluate fcn32_vgg16_pascal_voc
python eval.py --model fcn32s --backbone vgg16 --dataset pascal_voc
  • Multi-GPU evaluating
# for example, evaluate fcn32_vgg16_pascal_voc with 4 GPUs:
export NGPUS=4
python -m torch.distributed.launch --nproc_per_node=$NGPUS eval.py --model fcn32s --backbone vgg16 --dataset pascal_voc

Demo

cd ./scripts
#for new users:
python demo.py --model fcn32s_vgg16_voc --input-pic ../tests/test_img.jpg
#you should add 'test.jpg' by yourself
python demo.py --model fcn32s_vgg16_voc --input-pic ../datasets/test.jpg
.{SEG_ROOT}
├── scripts
│   ├── demo.py
│   ├── eval.py
│   └── train.py

Support

Model

DETAILS for model & backbone.

.{SEG_ROOT}
├── core
│   ├── models
│   │   ├── bisenet.py
│   │   ├── danet.py
│   │   ├── deeplabv3.py
│   │   ├── deeplabv3+.py
│   │   ├── denseaspp.py
│   │   ├── dunet.py
│   │   ├── encnet.py
│   │   ├── fcn.py
│   │   ├── pspnet.py
│   │   ├── icnet.py
│   │   ├── enet.py
│   │   ├── ocnet.py
│   │   ├── psanet.py
│   │   ├── cgnet.py
│   │   ├── espnet.py
│   │   ├── lednet.py
│   │   ├── dfanet.py
│   │   ├── ......

Dataset

You can run script to download dataset, such as:

cd ./core/data/downloader
python ade20k.py --download-dir ../datasets/ade
Dataset training set validation set testing set
VOC2012 1464 1449
VOCAug 11355 2857
ADK20K 20210 2000
Cityscapes 2975 500
COCO
SBU-shadow 4085 638
LIP(Look into Person) 30462 10000 10000
.{SEG_ROOT}
├── core
│   ├── data
│   │   ├── dataloader
│   │   │   ├── ade.py
│   │   │   ├── cityscapes.py
│   │   │   ├── mscoco.py
│   │   │   ├── pascal_aug.py
│   │   │   ├── pascal_voc.py
│   │   │   ├── sbu_shadow.py
│   │   └── downloader
│   │       ├── ade20k.py
│   │       ├── cityscapes.py
│   │       ├── mscoco.py
│   │       ├── pascal_voc.py
│   │       └── sbu_shadow.py

Result

  • PASCAL VOC 2012
Methods Backbone TrainSet EvalSet crops_size epochs JPU Mean IoU pixAcc
FCN32s vgg16 train val 480 60 47.50 85.39
FCN16s vgg16 train val 480 60 49.16 85.98
FCN8s vgg16 train val 480 60 48.87 85.02
FCN32s resnet50 train val 480 50 54.60 88.57
PSPNet resnet50 train val 480 60 63.44 89.78
DeepLabv3 resnet50 train val 480 60 60.15 88.36

Note: lr=1e-4, batch_size=4, epochs=80.

Overfitting Test

See TEST for details.

.{SEG_ROOT}
├── tests
│   └── test_model.py

To Do

  • add train script
  • remove syncbn
  • train & evaluate
  • test distributed training
  • fix syncbn (Why SyncBN?)
  • add distributed (How DIST?)

References

Comments
  • error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    thank you for your code. could you tell me what the mean of this code: cd awesome-semantic-segmentation-pytorch/core/nn python setup.py build develop. and the error is occur when i run it, could you help me how to solve it? 1111

    opened by wangxin1233 15
  • The question

    The question

    (The code is up to date.) 1.Is the loss small enough? (It is diverging during during this time->Loss: 2.0000~0.0200) There is no change at all in accuracy in epoch 1, 10,100 or 1000. I feel that processing is done with the weight of VGG16. Is there a problem with running the python3 code with python2?

    2.train.py utils->loss.py def _aux_forward(self, *inputs, **kwargs): *preds, target = tuple(inputs)<-----error *preds

    Is this okay with pred? def _aux_forward(self, *inputs, **kwargs): preds, target = tuple(inputs)

    opened by 82018303kazuma 15
  •  No module named 'core'

    No module named 'core'

    Thx for your work!but I have a problem that when I run demo.py, it retrun that "from core.utils.visualize import get_color_pallete ModuleNotFoundError: No module named 'core'" I have no idea about this. Could u fix it? and when I run train.py, it's OK.

    opened by NothingToSay99 14
  • train error

    train error

    python train.py

    Traceback (most recent call last): File "train.py", line 177, in trainer.train() File "train.py", line 147, in train if not args.Namespace: AttributeError: 'Namespace' object has no attribute 'Namespace' &&&&& pascal_voc.py

    from utils import makedirs

    cannot import name makedirs

    opened by 82018303kazuma 13
  • python setup.py error

    python setup.py error

    python setup.py build develop(pytorch ==1.1.0)

    2 errors detected in the compilation of "/tmp/tmpxft_000015ae_00000000-7_psa_cuda.cpp1.ii". error: command '/usr/local/cuda/bin/nvcc' failed with exit status 2

    I can not setup. I saw ”error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 #21” but I could not solve it

    opened by 82018303kazuma 12
  • some problems about

    some problems about "python setup.py build develop"

    I type "python setup.py build develop" and there are some errors I can not solve them.

    There is error information:

       ......
    4 errors detected in the compilation of 
    "C:/Users/35093/AppData/Local/Temp/tmpxft_00002d74_00000000-10_syncbn_cuda.cpp1.ii".
    syncbn_cuda.cu
    ninja: build stopped: subcommand failed.
    Traceback (most recent call last):
      File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 1400, in _run_ninja_build
        check=True)
      File "D:\anaconda3\envs\torch1\lib\subprocess.py", line 512, in run
        output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "setup.py", line 55, in <module>
        cmdclass={"build_ext": BuildExtension}
      File "D:\anaconda3\envs\torch1\lib\site-packages\setuptools\__init__.py", line 144, in setup
        return distutils.core.setup(**attrs)
      File "D:\anaconda3\envs\torch1\lib\distutils\core.py", line 148, in setup
        dist.run_commands()
      File "D:\anaconda3\envs\torch1\lib\distutils\dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "D:\anaconda3\envs\torch1\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "D:\anaconda3\envs\torch1\lib\distutils\command\build.py", line 135, in run
        self.run_command(cmd_name)
      File "D:\anaconda3\envs\torch1\lib\distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "D:\anaconda3\envs\torch1\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "D:\anaconda3\envs\torch1\lib\site-packages\setuptools\command\build_ext.py", line 87, in run
        _build_ext.run(self)
      File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 340, in run
        self.build_extensions()
      File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 580, in build_extensions
        build_ext.build_extensions(self)
      File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "D:\anaconda3\envs\torch1\lib\site-packages\setuptools\command\build_ext.py", line 208, in build_extension
        _build_ext.build_extension(self, ext)
      File "D:\anaconda3\envs\torch1\lib\distutils\command\build_ext.py", line 534, in build_extension
        depends=ext.depends)
      File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 562, in win_wrap_ninja_compile
        with_cuda=with_cuda)
      File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 1140, in _write_ninja_file_and_compile_objects
        error_prefix='Error compiling objects for extension')
      File "D:\anaconda3\envs\torch1\lib\site-packages\torch\utils\cpp_extension.py", line 1413, in _run_ninja_build
        raise RuntimeError(message)
    RuntimeError: Error compiling objects for extension
    

    my pc System: Win10 Python: 3.7.7 pytorch: 1.5.0 cudatoolkit : 10.1.243 cudnn: 7.6.5

    Can anyone help me? Thanks~

    opened by NoHateAnymore 10
  • undefined symbol: _Z23LeakyRelu_Backward_CUDAN2at6TensorES0_f

    undefined symbol: _Z23LeakyRelu_Backward_CUDAN2at6TensorES0_f

    it occurs this error when I run train.py

    Traceback (most recent call last): File "scripts/train.py", line 17, in from core.data.dataloader import get_segmentation_dataset File "/mnt/v-yuxzho/code/awesome-semantic-segmentation-pytorch-master/core/init.py", line 1, in from . import nn, models, utils, data File "/mnt/v-yuxzho/code/awesome-semantic-segmentation-pytorch-master/core/nn/init.py", line 2, in from .sync_bn.syncbn import * File "/mnt/v-yuxzho/code/awesome-semantic-segmentation-pytorch-master/core/nn/sync_bn/syncbn.py", line 17, in from .functions import * File "/mnt/v-yuxzho/code/awesome-semantic-segmentation-pytorch-master/core/nn/sync_bn/functions.py", line 15, in from core.nn.sync_bn import lib File "/mnt/v-yuxzho/code/awesome-semantic-segmentation-pytorch-master/core/nn/sync_bn/lib/init.py", line 19, in build_directory=gpu_path, verbose=False) File "/mnt/yueshangGu/Apps/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 645, in load is_python_module) File "/mnt/yueshangGu/Apps/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 825, in _jit_compile return _import_module_from_library(name, build_directory, is_python_module) File "/mnt/yueshangGu/Apps/anaconda3/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 968, in _import_module_from_library return imp.load_module(module_name, file, path, description) File "/mnt/yueshangGu/Apps/anaconda3/lib/python3.7/imp.py", line 242, in load_module return load_dynamic(name, filename, file) File "/mnt/yueshangGu/Apps/anaconda3/lib/python3.7/imp.py", line 342, in load_dynamic return _load(spec) ImportError: /mnt/v-yuxzho/code/awesome-semantic-segmentation-pytorch-master/core/nn/sync_bn/lib/gpu/sync_gpu.so: undefined symbol: _Z23LeakyRelu_Backward_CUDAN2at6TensorES0_f

    opened by Yuxiang1995 10
  • Validation pixAcc: 1.000, mIoU: 0.500

    Validation pixAcc: 1.000, mIoU: 0.500

    image hi,thank you for your code,now I want to train my own data by pspnet。 this is my first time to use pythorch, and I have added my own data.py in dataloader file. I don't know why pa is always 1.00 and miou is 0.5. Could you give me a hand? thank you very much.

    opened by wangxin1233 9
  • TypeError: forward() takes 3 positional arguments but 4 were given

    TypeError: forward() takes 3 positional arguments but 4 were given

    Hi, @Tramac : I use command as follow:

    CUDA_VISIBLE_DEVICES=4,5,6,7 python train.py --model psp --backbone resnet50 --dataset coco
    

    then i got error

    loading annotations into memory...
    Done (t=2.22s)
    creating index...
    index created!
    Starting Epoch: 0, Total Epochs: 60
    Traceback (most recent call last):
      File "train.py", line 248, in <module>
        trainer.train()
      File "train.py", line 183, in train
        loss = self.criterion(outputs, targets)
      File "/home/luqi/anaconda3/envs/pytorch-1.0/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
        result = self.forward(*input, **kwargs)
      File "/data/luqi/guojin/codes/awesome-semantic-segmentation-pytorch/core/utils/parallel.py", line 81, in forward
        outputs = criterion_parallel_apply(replicas, inputs, targets, kwargs)
      File "/data/luqi/guojin/codes/awesome-semantic-segmentation-pytorch/core/utils/parallel.py", line 163, in criterion_parallel_apply
        raise output
      File "/data/luqi/guojin/codes/awesome-semantic-segmentation-pytorch/core/utils/parallel.py", line 139, in _worker
        output = module(*(list(input) + [target]), **kwargs)
      File "/home/luqi/anaconda3/envs/pytorch-1.0/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
        result = self.forward(*input, **kwargs)
      File "/data/luqi/guojin/codes/awesome-semantic-segmentation-pytorch/core/nn/loss.py", line 32, in forward
        return super(MixSoftmaxCrossEntropyLoss, self).forward(*inputs)
    TypeError: forward() takes 3 positional arguments but 4 were given
    

    I dont quite understand what it means. Can you help me?

    opened by dekura 9
  • run demo.py RuntimeError: invalid argument 6:

    run demo.py RuntimeError: invalid argument 6:

    thanks for your help I just finished training and got the psanet model,then I just wanna run demo.py an error occurred ,can you help me to fix it again ,by the way os:win10 cuda10 pytorch1.1 C:\Users\123\Anaconda3\python.exe E:/awesome-semantic-segmentation-pytorch-master/awesome-semantic-segmentation-pytorch-master/scripts/demo.py C:\Users\123/.torch/models\psanet_resnet50_pascal_voc.pth Finished loading model! Traceback (most recent call last): File "E:/awesome-semantic-segmentation-pytorch-master/awesome-semantic-segmentation-pytorch-master/scripts/demo.py", line 58, in demo(args) File "E:/awesome-semantic-segmentation-pytorch-master/awesome-semantic-segmentation-pytorch-master/scripts/demo.py", line 49, in demo output = model(images) File "C:\Users\123\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call result = self.forward(*input, **kwargs) File "E:\awesome-semantic-segmentation-pytorch-master\awesome-semantic-segmentation-pytorch-master\core\models\psanet.py", line 48, in forward x = self.head(c4) File "C:\Users\123\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call result = self.forward(*input, **kwargs) File "E:\awesome-semantic-segmentation-pytorch-master\awesome-semantic-segmentation-pytorch-master\core\models\psanet.py", line 72, in forward global_feature = self.psa(x) File "C:\Users\123\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call result = self.forward(*input, **kwargs) File "E:\awesome-semantic-segmentation-pytorch-master\awesome-semantic-segmentation-pytorch-master\core\models\psanet.py", line 88, in forward collect_fm = self.collect_attention(x) File "C:\Users\123\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 493, in call result = self.forward(*input, **kwargs) File "E:\awesome-semantic-segmentation-pytorch-master\awesome-semantic-segmentation-pytorch-master\core\models\psanet.py", line 110, in forward fm = torch.bmm(reduce_x, torch.softmax(attention, dim=1)) RuntimeError: invalid argument 6: wrong matrix size at C:/w/1/s/windows/pytorch/aten/src\THC/generic/THCTensorMathBlas.cu:494

    Process finished with exit code 1

    opened by ChaosZX 8
  • ImportError: No module named 'utils

    ImportError: No module named 'utils

    hello, I run it on pytorch1.0.1 python3.5, but it has some problem with import models. when I run python ade20k.py --download-dir ./datasets/ade, I got "ImportError: No module named 'utils'" How should I solve it? Thank you.

    opened by casiahnu 7
  • DenseASPP与官方代码似乎实现的不太一样(different implemantation with official code in DenseASPP)

    DenseASPP与官方代码似乎实现的不太一样(different implemantation with official code in DenseASPP)

    When I try to reimplement the DenseASPP, I refer to both the official code and this repo, but when it comes to DenseASPP head, I notice the order of conv and norm in DenseASPP block is different.

    In official code, it's norm->relu->conv->norm->relu->conv, in this repo, it's conv->norm->relu->conv->norm->relu.

    Just the preact form of conv and postact form of convolution, is that mean the postact form performs better in semantic segmentation after your experiment?

    opened by PoloWitty 0
  • Hello! Is there any group number?

    Hello! Is there any group number?

    I want to communicate with people, and learn about how to improve the precision.

    and recently, I run some models, and I got a great precision score. When I try to repro it but failed.

    opened by 459737087 0
  • Made CGNet scriptable

    Made CGNet scriptable

    In it's current state, you cannot convert CGNet to torchscript using torch.jit.script. This is due to how torchscript handles variable resolution. I have changed some of the control flow in forward and I have also modified the constructor of ContextGuidedBlock slightly to fix these problems. This will be useful for those who want to compile their model using TensoRT for embedded devices.

    opened by nassir90 1
  • Why the performance dropped by half when use pytorch1.5 or the newer version?

    Why the performance dropped by half when use pytorch1.5 or the newer version?

    I have trained a model on pytorch1.3. And when I applied the trained model on pytroch1.5 or the newer version to test the same test data, the performance dropped by half. The evaluation indicators used for testing are F1-score and IOU.

    opened by bai-shui 0
Owner
Data&Model&Loss
null
This repository contains numerical implementation for the paper Intertemporal Pricing under Reference Effects: Integrating Reference Effects and Consumer Heterogeneity.

This repository contains numerical implementation for the paper Intertemporal Pricing under Reference Effects: Integrating Reference Effects and Consumer Heterogeneity.

Hansheng Jiang 6 Nov 18, 2022
Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation)

Recall Loss for Semantic Segmentation (This repo implements the paper: Recall Loss for Semantic Segmentation) Download Synthia dataset The model uses

null 32 Sep 21, 2022
A concise but complete implementation of CLIP with various experimental improvements from recent papers

x-clip (wip) A concise but complete implementation of CLIP with various experimental improvements from recent papers Install $ pip install x-clip Usag

Phil Wang 515 Dec 26, 2022
A concise but complete implementation of CLIP with various experimental improvements from recent papers

x-clip (wip) A concise but complete implementation of CLIP with various experimental improvements from recent papers Install $ pip install x-clip Usag

Phil Wang 115 Dec 9, 2021
This project is based on RIFE and aims to make RIFE more practical for users by adding various features and design new models

This project is based on RIFE and aims to make RIFE more practical for users by adding various features and design new models. Because improving the PSNR index is not compatible with subjective effects, we hope this part of work and our academic research are independent of each other.

hzwer 190 Jan 8, 2023
Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch.

Faster R-CNN and Mask R-CNN in PyTorch 1.0 maskrcnn-benchmark has been deprecated. Please see detectron2, which includes implementations for all model

Facebook Research 9k Jan 4, 2023
DRLib:A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos.

DRLib:A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos A concise deep reinforcement learning libr

null 329 Jan 3, 2023
A clear, concise, simple yet powerful and efficient API for deep learning.

The Gluon API Specification The Gluon API specification is an effort to improve speed, flexibility, and accessibility of deep learning technology for

Gluon API 2.3k Dec 17, 2022
LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models

LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models. Developers can reproduce these SOTA methods and build their own methods.

TuZheng 405 Jan 4, 2023
ComputerVision - This repository aims at realized easy network architecture

ComputerVision This repository aims at realized easy network architecture Colori

DongDong 4 Dec 14, 2022
Scripts of Machine Learning Algorithms from Scratch. Implementations of machine learning models and algorithms using nothing but NumPy with a focus on accessibility. Aims to cover everything from basic to advance.

Algo-ScriptML Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The goal of this project is not t

Algo Phantoms 81 Nov 26, 2022
A collection of easy-to-use, ready-to-use, interesting deep neural network models

Interesting and reproducible research works should be conserved. This repository wraps a collection of deep neural network models into a simple and un

Aria Ghora Prabono 16 Jun 16, 2022
this is a lite easy to use virtual keyboard project for anyone to use

virtual_Keyboard this is a lite easy to use virtual keyboard project for anyone to use motivation I made this for this year's recruitment for RobEn AA

Mohamed Emad 3 Oct 23, 2021
A Lighting Pytorch Framework for Recommendation System, Easy-to-use and Easy-to-extend.

Torch-RecHub A Lighting Pytorch Framework for Recommendation Models, Easy-to-use and Easy-to-extend. 安装 pip install torch-rechub 主要特性 scikit-learn风格易用

Mincai Lai 67 Jan 4, 2023
Project Aquarium is a SUSE-sponsored open source project aiming at becoming an easy to use, rock solid storage appliance based on Ceph.

Project Aquarium Project Aquarium is a SUSE-sponsored open source project aiming at becoming an easy to use, rock solid storage appliance based on Cep

Aquarist Labs 73 Jul 21, 2022
UpChecker is a simple opensource project to host it fast on your server and check is server up, view statistic, get messages if it is down. UpChecker - just run file and use project easy

UpChecker UpChecker is a simple opensource project to host it fast on your server and check is server up, view statistic, get messages if it is down.

Yan 4 Apr 7, 2022
TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

This project is a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.

yifan liu 147 Dec 3, 2022
[CVPR 2022] Official PyTorch Implementation for "Reference-based Video Super-Resolution Using Multi-Camera Video Triplets"

Reference-based Video Super-Resolution (RefVSR) Official PyTorch Implementation of the CVPR 2022 Paper Project | arXiv | RealMCVSR Dataset This repo c

Junyong Lee 151 Dec 30, 2022
Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

ADE20k Semantic segmentation with MAE Getting started Install the mmsegmentation

null 97 Dec 17, 2022