A collection of loss functions for medical image segmentation

Overview

Loss functions for image segmentation

A collection of loss functions for medical image segmentation

@article{LossOdyssey,
title = {Loss Odyssey in Medical Image Segmentation},
journal = {Medical Image Analysis},
volume = {71},
pages = {102035},
year = {2021},
author = {Jun Ma and Jianan Chen and Matthew Ng and Rui Huang and Yu Li and Chen Li and Xiaoping Yang and Anne L. Martel}
doi = {https://doi.org/10.1016/j.media.2021.102035},
url = {https://www.sciencedirect.com/science/article/pii/S1361841521000815}
}

Take-home message: compound loss functions are the most robust losses, especially for the highly imbalanced segmentation tasks.

Some recent side evidence: the winner in MICCAI 2020 HECKTOR Challenge used DiceFocal loss; the winner and runner-up in MICCAI 2020 ADAM Challenge used DiceTopK loss.

Date First Author Title Conference/Journal
20210330 Suprosanna Shit and Johannes C. Paetzold clDice - a Novel Topology-Preserving Loss Function for Tubular Structure Segmentation (keras and pytorch) CVPR 2021
20210318 Xiaoling Hu Topology-Aware Segmentation Using Discrete Morse Theory arxiv ICLR 2021
20210211 Hoel Kervadec Beyond pixel-wise supervision: semantic segmentation with higher-order shape descriptors Submitted to MIDL 2021
20210210 Rosana EL Jurdi A Surprisingly Effective Perimeter-based Loss for Medical Image Segmentation Submitted to MIDL 2021
20201222 Zeju Li Analyzing Overfitting Under Class Imbalance in Neural Networks for Image Segmentation TMI
20210129 Nick Byrne A Persistent Homology-Based Topological Loss Function for Multi-class CNN Segmentation of Cardiac MRI arxiv STACOM 2020
20201019 Hyunseok Seo Closing the Gap Between Deep Neural Network Modeling and Biomedical Decision-Making Metrics in Segmentation via Adaptive Loss Functions TMI
20200929 Stefan Gerl A Distance-Based Loss for Smooth and Continuous Skin Layer Segmentation in Optoacoustic Images MICCAI 2020
20200821 Nick Byrne A persistent homology-based topological loss function for multi-class CNN segmentation of cardiac MRI arxiv STACOM
20200720 Boris Shirokikh Universal Loss Reweighting to Balance Lesion Size Inequality in 3D Medical Image Segmentation arxiv (pytorch) MICCAI 2020
20200708 Gonglei Shi Marginal loss and exclusion loss for partially supervised multi-organ segmentation (arXiv) MedIA
20200706 Yuan Lan An Elastic Interaction-Based Loss Function for Medical Image Segmentation (pytorch) (arXiv) MICCAI 2020
20200615 Tom Eelbode Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index TMI
20200605 Guotai Wang Noise-robust Dice loss: A Noise-robust Framework for Automatic Segmentation of COVID-19 Pneumonia Lesions from CT Images (pytorch) TMI
202004 J. H. Moltz Contour Dice coefficient (CDC) Loss: Learning a Loss Function for Segmentation: A Feasibility Study ISBI
201912 Yuan Xue Shape-Aware Organ Segmentation by Predicting Signed Distance Maps (arxiv) (pytorch) AAAI 2020
201912 Xiaoling Hu Topology-Preserving Deep Image Segmentation (paper) (pytorch) NeurIPS
201910 Shuai Zhao Region Mutual Information Loss for Semantic Segmentation (paper) (pytorch) NeurIPS 2019
201910 Shuai Zhao Correlation Maximized Structural Similarity Loss for Semantic Segmentation (paper) arxiv
201908 Pierre-AntoineGanaye Removing Segmentation Inconsistencies with Semi-Supervised Non-Adjacency Constraint (paper) (official pytorch) Medical Image Analysis
201906 Xu Chen Learning Active Contour Models for Medical Image Segmentation (paper) (official-keras) CVPR 2019
20190422 Davood Karimi Reducing the Hausdorff Distance in Medical Image Segmentation with Convolutional Neural Networks (paper) (pytorch) TMI 201907
20190417 Francesco Caliva Distance Map Loss Penalty Term for Semantic Segmentation (paper) MIDL 2019
20190411 Su Yang Major Vessel Segmentation on X-ray Coronary Angiography using Deep Networks with a Novel Penalty Loss Function (paper) MIDL 2019
20190405 Boah Kim Multiphase Level-Set Loss for Semi-Supervised and Unsupervised Segmentation with Deep Learning (paper) arxiv
201901 Seyed Raein Hashemi Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection (paper) IEEE Access
201812 Hoel Kervadec Boundary loss for highly unbalanced segmentation (paper), (pytorch 1.0) MIDL 2019
201810 Nabila Abraham A Novel Focal Tversky loss function with improved Attention U-Net for lesion segmentation (paper) (keras) ISBI 2019
201809 Fabian Isensee CE+Dice: nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation (paper) arxiv
20180831 Ken C. L. Wong 3D Segmentation with Exponential Logarithmic Loss for Highly Unbalanced Object Sizes (paper) MICCAI 2018
20180815 Wentao Zhu Dice+Focal: AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy (arxiv) (pytorch) Medical Physics
201806 Javier Ribera Weighted Hausdorff Distance: Locating Objects Without Bounding Boxes (paper), (pytorch) CVPR 2019
201805 Saeid Asgari Taghanaki Combo Loss: Handling Input and Output Imbalance in Multi-Organ Segmentation (arxiv) (keras) Computerized Medical Imaging and Graphics
201709 S M Masudur Rahman AL ARIF Shape-aware deep convolutional neural network for vertebrae segmentation (paper) MICCAI 2017 Workshop
201708 Tsung-Yi Lin Focal Loss for Dense Object Detection (paper), (code) ICCV, TPAMI
20170711 Carole Sudre Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations (paper) DLMIA 2017
20170703 Lucas Fidon Generalised Wasserstein Dice Score for Imbalanced Multi-class Segmentation using Holistic Convolutional Networks (paper) MICCAI 2017 BrainLes
201705 Maxim Berman The Lovász-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks (paper), (code) CVPR 2018
201701 Seyed Sadegh Mohseni Salehi Tversky loss function for image segmentation using 3D fully convolutional deep networks (paper) MICCAI 2017 MLMI
201612 Md Atiqur Rahman Optimizing Intersection-Over-Union in Deep Neural Networks for Image Segmentation (paper) 2016 International Symposium on Visual Computing
201608 Michal Drozdzal "Dice Loss (without square)" The Importance of Skip Connections in Biomedical Image Segmentation (arxiv) DLMIA 2016
201606 Fausto Milletari "Dice Loss (with square)" V-net: Fully convolutional neural networks for volumetric medical image segmentation (arxiv), (caffe code) International Conference on 3D Vision
201605 Zifeng Wu TopK loss Bridging Category-level and Instance-level Semantic Image Segmentation (paper) arxiv
201511 Tom Brosch "Sensitivity-Specifity loss" Deep Convolutional Encoder Networks for Multiple Sclerosis Lesion Segmentation (paper) (code) MICCAI 2015
201505 Olaf Ronneberger "Weighted cross entropy" U-Net: Convolutional Networks for Biomedical Image Segmentation (paper) MICCAI 2015
201309 Gabriela Csurka What is a good evaluation measure for semantic segmentation? (paper) BMVA 2013

Most of the corresponding tensorflow code can be found here.

Comments
  • about multi-organ segmentation loss: Dice+Focal loss

    about multi-organ segmentation loss: Dice+Focal loss

    Hi, Jun, thank you very much for your valuable work. When you perform multi-organ segmentation on nnUNet, you observed that the combine of Dice loss and Focal loss achieved the best DSC. Can you share your parameters used in Focal loss? Such as the alpha and gamma and learning rate. Many thanks, waiting for your reply.

    opened by GuobinZhangTJU 8
  • Will torch.no_grad() make it impossible to update the weight and only calculate the value for printing? ? ? ?

    Will torch.no_grad() make it impossible to update the weight and only calculate the value for printing? ? ? ?

    Thanks for collecting and sharing the loss function work. There are a lot of torch.no_grad() used in the code. Will this cause the gradient to be truncated, making it impossible to update the weights?

    opened by yanghedada 7
  • HausdorffERLoss is not differentiable

    HausdorffERLoss is not differentiable

    I wrote a doctest for the erosion based hausdorff loss and I found it is not differentiable:

        Examples:
            >>> image_preds = torch.tensor([[
            ...    [0, 0.1, 0.1, 0],
            ...    [0, 0.3, 0.6, 0],
            ...    [0, 0.4, 0.8, 0],
            ...    [0, 0, 0, 0],
            ... ]], requires_grad=True)[None]
            >>> image_gt = torch.tensor([[
            ...    [0, 1, 0, 0],
            ...    [1, 1, 1, 0],
            ...    [1, 1, 1, 0],
            ...    [0, 0, 0, 0],
            ... ]])[None]
            >>> hdloss = HausdorffERLoss(from_logits=False)
            >>> hdloss(image_preds, image_gt)
            tensor(0.0625)
            >>> hdloss(image_preds, image_gt).backward()
    

    I would suspect something is missing from your implementation.

    opened by shijianjian 6
  • about gradient

    about gradient

    I'm sorry to bother you.In SegLoss/losses_pytorch/hausdorff.py,I find function distance_field use torch.no_grad,this wil make this fuction have no gradient, Will this have an impact on training?I don't know much about the impact.Can you explain it for me? Thanks.

    opened by rookiez7 5
  • Data preprocessing

    Data preprocessing

    Hi, @JunMa11,

    Thanks for your great work. I have a question about data preprocessing. As you've provided the detailed splits for the datasets. Could you further provide the code for liver CT dataset processing, so we can compare our results with yours directly?

    Best, Xi

    opened by xifang001 5
  • about DC_and_HD_loss

    about DC_and_HD_loss

    Hi, Jun, I want to use DC+HD as the loss to train a multi-organ segmentation model, but I meet two problems:

    1. in nnUNetTrainer_DiceHDBinary.py: from nnunet.training.loss_functions.boundary_loss import DC_and_HDBinary_loss, but there is no DC_and_HDBinary_loss in boundary_loss.py, only has DC_and_HD_loss, are they the same?
    2. in the defination of DC_and_HD_loss def forward(self, net_output, target): dc_loss = self.dc(net_output, target) hd_loss = self.hd(net_output, target) if self.aggregate == "sum": with torch.no_grad(): alpha = hd_loss / (dc_loss + 1e-5 ) result = alpha * dc_loss + hd_loss else: raise NotImplementedError("nah son") return result Here, alpha = hd_loss / (dc_loss + 1e-5 ), result = alpha * dc_loss + hd_loss, alpha * dc_loss = hd_loss / (dc_loss + 1e-5 ) * dc_loss, where 1e-5 is too small which can be ignore, then the result will be hd_loss , the final result will equals to 2*hd_loss, not hd_loss+dc_loss, isn't it? Thanks, waiting for your reply~
    opened by GuobinZhangTJU 5
  • Which one is the loss from

    Which one is the loss from "Topology-Preserving Deep Image Segmentation" paper ?

    Hi,

    Thanks for this nice repo! I'm looking for the loss from this paper "Topology-Preserving Deep Image Segmentation" it seems to appear your table so I guess you've implemented it but I did not find it. Would you mind pointing me the correct loss function please?

    Thank you very much,

    Bests

    opened by Optimox 4
  • Focal Loss

    Focal Loss

    Hello, @JunMa11

    I want to know whether focal loss is suitable for 3D image segmentation or not,and for segmentation with only two classifications of background and foreground,whether the network can be single-channel output and use focal loss.

    Hope your help, thank you!

    opened by ruiruian 3
  • Dataset issue, can help!

    Dataset issue, can help!

    Hey Jun. @JunMa11 I hope you are fine. I thought I shall ask you as you have already tried the dataset conversion from Fabian's repo. So, I download the cardiac data from ACDC, they have training repo, but when I ran the code (for ACDC data-preparation) fromhis repo, it's throwing the error. I can't use the Hackathon data, as it has got only 30 set. Can you please clarify whether it's the right process? I just need to convert the data, ans strip the 4D to 3D.

    opened by Sayantan17 3
  • when `apply_nonlin=None` calculation of GDiceLoss fails because softmax_output is not created

    when `apply_nonlin=None` calculation of GDiceLoss fails because softmax_output is not created

    when apply_nonlin=None calculation of GDiceLoss fails:

    Traceback (most recent call last):
      File "path/loser.py", line 181, in <module>
        loss = criterion(prediction, label)
      File "path/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
        result = self.forward(*input, **kwargs)
      File "path/SegLoss/losses_pytorch/dice_loss.py", line 123, in forward
        intersection: torch.Tensor = w * einsum("bcxyz, bcxyz->bc", softmax_output, y_onehot)
    UnboundLocalError: local variable 'softmax_output' referenced before assignment
    
    opened by neuronflow 3
  • [Question] Why are losses negative?

    [Question] Why are losses negative?

    Many losses in the end return -loss , e.g. SoftDiceLoss or Tversky: https://github.com/JunMa11/SegLoss/blob/131663994df22efb907e7ae897a39efe628518fc/losses_pytorch/dice_loss.py#L291

    Why are the losses returned negative?

    opened by neuronflow 3
  • why is there no resolution input in the distance_map computing(scipy.ndimage.distance_transform_edt)?

    why is there no resolution input in the distance_map computing(scipy.ndimage.distance_transform_edt)?

    Hello, Thanks for sharing the code. it helps me a lot. The paper (loss odyssey ) is also great for me to know more about loss in medical segmentation. I have found that in your code, the resolution is not considered in distance_map computing. However, it is used herehttps://github.com/LIVIAETS/boundary-loss. Why is the resolution not be used here? Looking forward to your reply. Thanks a lot.

    opened by SummerZ2020 0
  • Hello, can you provide the ASOCA dataset?

    Hello, can you provide the ASOCA dataset?

    I am interested in this challenge, but i got nothing from the organizer. It would be greatly helpful if you can provide the dataset. My email is [email protected].

    opened by wlsdzyzl 0
  • About Sensitivity Specificity loss function implementation

    About Sensitivity Specificity loss function implementation

    Hello, I have a doubt about Sensitivity Specificity loss function's implementation. image So I think that the correct way to translate this Sensitivity into **prediction ** and **ground_truth ** terms for a loss function would be similar to: true_positives = prediction*ground_truth false_negatives = (1-prediction)*ground_truth image But in the implementation: Sensitivity = (square(ground_truth-prediction) * (1-one_hot) ) / (1-one_hot) How should I interpret its expressions(The same occurs with the Specificity )?

    opened by Liqq1 0
  • multi focal loss alpha

    multi focal loss alpha

    Hi, I'd like to ask Focal Loss with multiple classifications. So for example, I have background, C1,C2,C3, four categories. If I set alpha to [0.25, 1,1,0.5], does that mean that my background weight is 0.25, then the foreground three classes are weighted 1,1,0.5, and C1 and C2 have the same weight and are twice as heavy as C3

    opened by kimjisoo12 0
  • multi focal loss alpha

    multi focal loss alpha

    Hi, I'd like to ask Focal Loss with multiple classifications. So for example, I have background, C1,C2,C3, four categories. If I set alpha to [0.25, 1,1,0.5], does that mean that my background weight is 0.25, then the foreground three classes are weighted 1,1,0.5, and C1 and C2 have the same weight and are twice as heavy as C3

    opened by kimjisoo12 0
  • digital-copyright

    digital-copyright

    Hi JunMa11!👋, I added this optional feature to digitally sign you source-code and track it on a blockchain node should you ever be audited or experience a software supply-chain attack. Simply compare the byte encrypted signature on your .git binary with the hash written to your immutable blockchain node. If they ever differ you should escalate. See the JunMa11-digital-copyright for complete instructions on accessing your hash.. Feel free to contact me directly to review any questions before accepting. ~~Best: [email protected]

    opened by JudeSafo 1
Owner
Jun
Jun
Pytorch Code for "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation"

Medical-Transformer Pytorch Code for the paper "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation" About this repo: This repo

Jeya Maria Jose 615 Dec 25, 2022
Semi Supervised Learning for Medical Image Segmentation, a collection of literature reviews and code implementations.

Semi-supervised-learning-for-medical-image-segmentation. Recently, semi-supervised image segmentation has become a hot topic in medical image computin

Healthcare Intelligence Laboratory 1.3k Jan 3, 2023
Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target image;

NanYoMy 13 Oct 9, 2022
Copy Paste positive polyp using poisson image blending for medical image segmentation

Copy Paste positive polyp using poisson image blending for medical image segmentation According poisson image blending I've completely used it for bio

Phạm Vũ Hùng 2 Oct 19, 2021
The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

MIC-DKFZ 1.2k Jan 4, 2023
Build a medical knowledge graph based on Unified Language Medical System (UMLS)

UMLS-Graph Build a medical knowledge graph based on Unified Language Medical System (UMLS) Requisite Install MySQL Server 5.6 and import UMLS data int

Donghua Chen 6 Dec 25, 2022
An implementation for the loss function proposed in Decoupled Contrastive Loss paper.

Decoupled-Contrastive-Learning This repository is an implementation for the loss function proposed in Decoupled Contrastive Loss paper. Requirements P

Ramin Nakhli 71 Dec 4, 2022
Implement of "Training deep neural networks via direct loss minimization" in PyTorch for 0-1 loss

This is the implementation of "Training deep neural networks via direct loss minimization" published at ICML 2016 in PyTorch. The implementation targe

Cuong Nguyen 1 Jan 18, 2022
git《Investigating Loss Functions for Extreme Super-Resolution》(CVPR 2020) GitHub:

Investigating Loss Functions for Extreme Super-Resolution NTIRE 2020 Perceptual Extreme Super-Resolution Submission. Our method ranked first and secon

Sejong Yang 0 Oct 17, 2022
This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

TransUNet This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation Usage

null 1.4k Jan 4, 2023
CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation

CoTr: Efficient 3D Medical Image Segmentation by bridging CNN and Transformer This is the official pytorch implementation of the CoTr: Paper: CoTr: Ef

null 218 Dec 25, 2022
[CVPR'21] FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space

FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space by Quande Liu, Cheng Chen, Ji

Quande Liu 178 Jan 6, 2023
Medical Image Segmentation using Squeeze-and-Expansion Transformers

Medical Image Segmentation using Squeeze-and-Expansion Transformers Introduction This repository contains the code of the IJCAI'2021 paper 'Medical Im

askerlee 172 Dec 20, 2022
The codes for the work "Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation"

Swin-Unet The codes for the work "Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation"(https://arxiv.org/abs/2105.05537). A validatio

null 869 Jan 7, 2023
Segmentation for medical image.

EfficientSegmentation Introduction EfficientSegmentation is an open source, PyTorch-based segmentation framework for 3D medical image. Features A whol

null 68 Nov 28, 2022
A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

A PyTorch implementation of V-Net Vnet is a PyTorch implementation of the paper V-Net: Fully Convolutional Neural Networks for Volumetric Medical Imag

Matthew Macy 606 Dec 21, 2022
A pytorch-based deep learning framework for multi-modal 2D/3D medical image segmentation

A 3D multi-modal medical image segmentation library in PyTorch We strongly believe in open and reproducible deep learning research. Our goal is to imp

Adaloglou Nikolas 1.2k Dec 27, 2022
Bayesian Optimization Library for Medical Image Segmentation.

bayesmedaug: Bayesian Optimization Library for Medical Image Segmentation. bayesmedaug optimizes your data augmentation hyperparameters for medical im

Şafak Bilici 7 Feb 10, 2022
A keras-based real-time model for medical image segmentation (CFPNet-M)

CFPNet-M: A Light-Weight Encoder-Decoder Based Network for Multimodal Biomedical Image Real-Time Segmentation This repository contains the implementat

null 268 Nov 27, 2022