PyTorch implementation of UNet++ (Nested U-Net).

Overview

PyTorch implementation of UNet++ (Nested U-Net)

MIT License

This repository contains code for a image segmentation model based on UNet++: A Nested U-Net Architecture for Medical Image Segmentation implemented in PyTorch.

[NEW] Add support for multi-class segmentation dataset.

[NEW] Add support for PyTorch 1.x.

Requirements

  • PyTorch 1.x or 0.41

Installation

  1. Create an anaconda environment.
conda create -n=<env_name> python=3.6 anaconda
conda activate <env_name>
  1. Install PyTorch.
conda install pytorch torchvision cudatoolkit=10.1 -c pytorch
  1. Install pip packages.
pip install -r requirements.txt

Training on 2018 Data Science Bowl dataset

  1. Download dataset from here to inputs/ and unzip. The file structure is the following:
inputs
└── data-science-bowl-2018
    ├── stage1_train
    |   ├── 00ae65...
    │   │   ├── images
    │   │   │   └── 00ae65...
    │   │   └── masks
    │   │       └── 00ae65...            
    │   ├── ...
    |
    ...
  1. Preprocess.
python preprocess_dsb2018.py
  1. Train the model.
python train.py --dataset dsb2018_96 --arch NestedUNet
  1. Evaluate.
python val.py --name dsb2018_96_NestedUNet_woDS

(Optional) Using LovaszHingeLoss

  1. Clone LovaszSoftmax from bermanmaxim/LovaszSoftmax.
git clone https://github.com/bermanmaxim/LovaszSoftmax.git
  1. Train the model with LovaszHingeLoss.
python train.py --dataset dsb2018_96 --arch NestedUNet --loss LovaszHingeLoss

Training on original dataset

Make sure to put the files as the following structure (e.g. the number of classes is 2):

inputs
└── <dataset name>
    ├── images
    |   ├── 0a7e06.jpg
    │   ├── 0aab0a.jpg
    │   ├── 0b1761.jpg
    │   ├── ...
    |
    └── masks
        ├── 0
        |   ├── 0a7e06.png
        |   ├── 0aab0a.png
        |   ├── 0b1761.png
        |   ├── ...
        |
        └── 1
            ├── 0a7e06.png
            ├── 0aab0a.png
            ├── 0b1761.png
            ├── ...
  1. Train the model.
python train.py --dataset <dataset name> --arch NestedUNet --img_ext .jpg --mask_ext .png
  1. Evaluate.
python val.py --name <dataset name>_NestedUNet_woDS

Results

DSB2018 (96x96)

Here is the results on DSB2018 dataset (96x96) with LovaszHingeLoss.

Model IoU Loss
U-Net 0.839 0.365
Nested U-Net 0.842 0.354
Nested U-Net w/ Deepsupervision 0.843 0.362
Comments
  • TypeError: 'NoneType' object is not subscriptable

    TypeError: 'NoneType' object is not subscriptable

    File "D:\pytorch-nested-unet-master\dataset.py", line 63, in getitem img_id + self.mask_ext), cv2.IMREAD_GRAYSCALE)[..., None]) TypeError: 'NoneType' object is not subscriptable

    i m getting this error while training model, Can anyone help??

    opened by manvirvirk 30
  • ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters.

    ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters.

    pc:~/unets/pytorch-nested-unet-master$ python train.py --dataset dsb2018_96 --arch NestedUNet Config ----- name: dsb2018_96_NestedUNet_woDS arch: NestedUNet deepsupervision: False dataset: dsb2018_96 input_channels: 3 image_ext: png mask_ext: png aug: False loss: BCEDiceLoss epochs: 10000 early_stop: 20 batch_size: 16 optimizer: Adam lr: 0.0003 momentum: 0.9 weight_decay: 0.0001 nesterov: False

    Traceback (most recent call last): File "train.py", line 301, in main() File "train.py", line 222, in main train_test_split(img_paths, mask_paths, test_size=0.2, random_state=41) File "/home/jiyue/anaconda3/envs/judy/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 2100, in train_test_split default_test_size=0.25) File "/home/jiyue/anaconda3/envs/judy/lib/python3.7/site-packages/sklearn/model_selection/_split.py", line 1782, in _validate_shuffle_split train_size) ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters

    opened by JiyueWang 12
  • error: NameError: name 'joblib' is not defined

    error: NameError: name 'joblib' is not defined

    File "train.py", line 300, in main() File "train.py", line 206, in main joblib.dump(args, 'models/%s/args.pkl' %args.name) NameError: name 'joblib' is not defined i m getting above error?

    opened by manvirvirk 7
  • IoU only 0.843? How can we achieve 92.37%±0.98 IoU showed in UNet++ paper?

    IoU only 0.843? How can we achieve 92.37%±0.98 IoU showed in UNet++ paper?

    image <UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation> shows a high IoU of DSB2018 up to 92.37%. This repo has any differences? Or do we need some tricks? have you think about it? Hoping for reply. Thanks

    opened by LiuFG 5
  • Question about

    Question about "ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters".

    When I use "python train.py --dataset dsb2018_96 --arch NestedUNet" to run the code,why then it turns out"ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters."?

    opened by wxyWendy 3
  • error when run test.py: CUDNN_STATUS_EXECUTION_FAILED

    error when run test.py: CUDNN_STATUS_EXECUTION_FAILED

    hi, i got a error 'CUDNN_STATUS_EXECUTION_FAILED' when run the test.py. Here is my environment: python3.6, cuda=9.0, cudnn=7.6.5, pytorch=0.4.0. I carefully followed the requirement.txt, but still got this error, any suggestion? Thank you

    opened by godlovehill 3
  • When I run the train.py,the results are

    When I run the train.py,the results are "loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000"?

    When I run the train.py,why the results are "loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000"?I run on the dataset dsb2018_96. loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000 Epoch [93/100] 100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.80it/s, loss=nan, iou=5.04e-10] 100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.14it/s, loss=nan, iou=5.76e-10] loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000 Epoch [94/100] 100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.72it/s, loss=nan, iou=5.01e-10] 100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.95it/s, loss=nan, iou=5.76e-10] loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000 Epoch [95/100] 100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.77it/s, loss=nan, iou=5.14e-10] 100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.63it/s, loss=nan, iou=5.76e-10] loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000 Epoch [96/100] 100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.76it/s, loss=nan, iou=5.06e-10] 100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.84it/s, loss=nan, iou=5.76e-10] loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000 Epoch [97/100] 100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.74it/s, loss=nan, iou=5.11e-10] 100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 37.12it/s, loss=nan, iou=5.76e-10] loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000 Epoch [98/100] 100%|███████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.79it/s, loss=nan, iou=5.05e-10] 100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 36.90it/s, loss=nan, iou=5.76e-10] loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000 Epoch [99/100] 100%|████████████████████████████████████████████████████████████| 33/33 [00:01<00:00, 18.77it/s, loss=nan, iou=5.2e-10] 100%|█████████████████████████████████████████████████████████████| 9/9 [00:00<00:00, 37.04it/s, loss=nan, iou=5.76e-10] loss nan - iou 0.0000 - val_loss nan - val_iou 0.0000

    opened by wxyWendy 2
  • early stopping , could not train after 10 epoch

    early stopping , could not train after 10 epoch

    100%|██████████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.95s/it, loss=0.671, iou=0.63] 100%|██████████████████████████████████████████████████████████| 1/1 [00:01<00:00, 1.71s/it, loss=0.788, iou=3.97e-10] loss 0.6712 - iou 0.6299 - val_loss 0.7878 - val_iou 0.0000 => early stopping

    opened by manvirvirk 2
  • deploy on opencv

    deploy on opencv

    Mat img = imread("test.png");
    float scale = 1.0 / 255;
    Size inputSize(256, 256);
    
    Mat inputBolb = cv::dnn::blobFromImage(img, scale, inputSize, Scalar(), true, true);
    
    dnn::Net my_net = cv::dnn::readNetFromONNX("unet_model.onnx");
    my_net.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
    my_net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
    
    my_net.setInput(inputBolb);
    Mat result = my_net.forward();
    

    i try to deploy this model on opencv(after convert pth to onnx, two classes), it seems that result is single channel, so it can only get one label, but label is two classes, so how to get the two classes, any suggestioin?

    opened by godlovehill 1
  •  Please change the validation code.

    Please change the validation code.

    If the code is not modified, the same output file is created.

    https://github.com/4uiiurz1/pytorch-nested-unet/blob/1424f72b7ac3816e450a1c596450e3e68e678f4a/val.py#L105

    (output[i, c] * 255).astype('uint8'))

    opened by tjdalsckd 1
  • Results

    Results

    Nice code, thanks for sharing.

    As you did not show results, in your opinion where nested-unet/unet++ stands in precision between standard resnet, densenet, squeezenet etc?

    Does it perform much better?

    opened by evaldsurtans 1
  • What is the correct format for training on custom data?

    What is the correct format for training on custom data?

    Hi, I see in the description the following layout is required for training on a custom dataset:

    inputs
    └── <dataset name>
        ├── images
        |   ├── 0a7e06.jpg
        │   ├── 0aab0a.jpg
        │   ├── 0b1761.jpg
        │   ├── ...
        |
        └── masks
            ├── 0
            |   ├── 0a7e06.png
            |   ├── 0aab0a.png
            |   ├── 0b1761.png
            |   ├── ...
            |
            └── 1
                ├── 0a7e06.png
                ├── 0aab0a.png
                ├── 0b1761.png
                ├── ...
    
    

    Let's say 0a7e06.png has one instance of class 0 and no instances of class 1. My question is, is a blank image in the class 1 folder named 0a7e06.png required? Or is 0a7e06.png only required to exist in folder 0?

    Thanks

    opened by Callum232310 0
  • albumentations

    albumentations

    Hello, may I ask which version of albumentations did you use in your code? Now some features of this library have changed, so I do not know which version to use. Thank you very much!

    opened by odyle-1103 0
  • crime detection project

    crime detection project

    hi everyone I need your help when I run my project I got this error I don't know how to solve ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters. I don't know what kind of error or why this occurs can u please help me Thank you

    opened by 2010030529 1
  • about deep supervision

    about deep supervision

    hi How do I choose type of deep supervision? I can't see this in your code. If I want to use type of ensemble for deep supervision how can I do? pruned too thank you

    opened by suh1030 0
  • Question about training the model

    Question about training the model

    I read your paper and the original of UNet++ and everything is clear to me apart from how I can I train the network using a new dataset. There is the option of deep supervision in which the nework takes the co-registered image pairs (each has size 256x256x6) concatenated and as output the [output1, output2, output3, output4], with each output dimension of 256x256x1. I would like to reproduce your experiments with deep supervision. It is clear to me what the input should be, but I fail to understand what the output should be. Specifically, how can I produce these 4 output matrices. In the dataset from Lebedev only one grayscale image was used as output, and each pixel of this image was produced from subtracting the two input images.

    opened by mpegia 0
Owner
4ui_iurz1
4ui_iurz1
U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection

The code for our newly accepted paper in Pattern Recognition 2020: "U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection."

Xuebin Qin 6.5k Jan 9, 2023
The open source code of SA-UNet: Spatial Attention U-Net for Retinal Vessel Segmentation.

SA-UNet: Spatial Attention U-Net for Retinal Vessel Segmentation(ICPR 2020) Overview This code is for the paper: Spatial Attention U-Net for Retinal V

Changlu Guo 151 Dec 28, 2022
RGBD-Net - This repository contains a pytorch lightning implementation for the 3DV 2021 RGBD-Net paper.

[3DV 2021] We propose a new cascaded architecture for novel view synthesis, called RGBD-Net, which consists of two core components: a hierarchical depth regression network and a depth-aware generator network.

Phong Nguyen Ha 4 May 26, 2022
A PyTorch Implementation of the Luna: Linear Unified Nested Attention

Unofficial PyTorch implementation of Luna: Linear Unified Nested Attention The quadratic computational and memory complexities of the Transformer’s at

Soohwan Kim 32 Nov 7, 2022
U^2-Net - Portrait matting This repository explores possibilities of using the original u^2-net model for portrait matting.

U^2-Net - Portrait matting This repository explores possibilities of using the original u^2-net model for portrait matting.

Dennis Bappert 104 Nov 25, 2022
The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

MIC-DKFZ 1.2k Jan 4, 2023
Neural networks applied in recognizing guitar chords using python, AutoML.NET with C# and .NET Core

Chord Recognition Demo application The demo application is written in C# with .NETCore. As of July 9, 2020, the only version available is for windows

Andres Mauricio Rondon Patiño 24 Oct 22, 2022
U-2-Net: U Square Net - Modified for paired image training of style transfer

U2-Net: U Square Net Modified for paired image training of style transfer This is an unofficial repo making use of the code which was made available b

Doron Adler 43 Oct 3, 2022
Implementation of Uformer, Attention-based Unet, in Pytorch

Uformer - Pytorch Implementation of Uformer, Attention-based Unet, in Pytorch. It will only offer the concat-cross-skip connection. This repository wi

Phil Wang 72 Dec 19, 2022
MIMO-UNet - Official Pytorch Implementation

MIMO-UNet - Official Pytorch Implementation This repository provides the official PyTorch implementation of the following paper: Rethinking Coarse-to-

Sungjin Cho 248 Jan 2, 2023
a reimplementation of Holistically-Nested Edge Detection in PyTorch

pytorch-hed This is a personal reimplementation of Holistically-Nested Edge Detection [1] using PyTorch. Should you be making use of this work, please

Simon Niklaus 375 Dec 6, 2022
Aggragrating Nested Transformer Official Jax Implementation

NesT is a simple method, which aggragrates nested local transformers on image blocks. The idea makes vision transformers attain better accuracy, data efficiency, and convergence on the ImageNet benchmark. NesT can be scaled to small datasets to match convnet accuracy.

Google Research 169 Dec 20, 2022
This is a file about Unet implemented in Pytorch

Unet this is an implemetion of Unet in Pytorch and it's architecture is as follows which is the same with paper of Unet component of Unet Convolution

Dragon 1 Dec 3, 2021
Using pytorch to implement unet network for liver image segmentation.

Using pytorch to implement unet network for liver image segmentation.

zxq 1 Dec 17, 2021
Implementation detail for paper "Multi-level colonoscopy malignant tissue detection with adversarial CAC-UNet"

Multi-level-colonoscopy-malignant-tissue-detection-with-adversarial-CAC-UNet Implementation detail for our paper "Multi-level colonoscopy malignant ti

CVSM Group -  email: czhu@bupt.edu.cn 84 Nov 22, 2022
Implementation of UNet on the Joey ML framework

Independent Research Project - Code Joey can be cloned from here https://github.com/devitocodes/joey/. Devito and other dependencies such as PyTorch a

Navjot Kukreja 1 Oct 21, 2021
Implementation of UNET architecture for Image Segmentation.

Semantic Segmentation using UNET This is the implementation of UNET on Carvana Image Masking Kaggle Challenge About the Dataset This dataset contains

Anushka agarwal 4 Dec 21, 2021
Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

Namt0d 9 Dec 7, 2022
Official Keras Implementation for UNet++ in IEEE Transactions on Medical Imaging and DLMIA 2018

UNet++: A Nested U-Net Architecture for Medical Image Segmentation UNet++ is a new general purpose image segmentation architecture for more accurate i

Zongwei Zhou 1.8k Jan 7, 2023