Context Axial Reverse Attention Network for Small Medical Objects Segmentation

Overview

CaraNet: Context Axial Reverse Attention Network for Small Medical Objects Segmentation

PWC

PWC

PWC

PWC

Result
This repository contains the implementation of a novel attention based network (CaraNet) to segment the polyp (CVC-T, CVC-ClinicDB, CVC-ColonDB, ETIS and Kvasir) and brain tumor (BraTS). The CaraNet show great overall segmentation performance (mean dice) on polyp and brain tumor, but also show great performance on small medical objects (small polyps and brain tumors) segmentation.

The technique report is here: CaraNet

Architecture of CaraNet

Backbone

We use Res2Net as our backbone.

Context module

We choose our CFP module as context module, and choose the dilation rate is 8. For the details of CFP module you can find here: CFPNet. The architecture of CFP module as shown in following figure:

Result

Axial Reverse Attention

As shown in architecture of CaraNet, the Axial Reverse Attention (A-RA) module contains two routes: 1) Reverse attention; 2) Axial-attention.

Installation & Usage

Enviroment

  • Enviroment: Python 3.6;
  • Install some packages:
conda install pytorch==1.1.0 torchvision==0.3.0 cudatoolkit=10.0 -c pytorch
conda install opencv-python pillow numpy matplotlib
  • Clone this repository
git clone https://github.com/AngeLouCN/CaraNet

Training

  • Download the training and texting dataset from this link: Experiment Dataset
  • Change the --train_path & --test_path in Train.py
  • Run Train.py
  • Testing dataset is ordered as follow:
|-- TestDataset
|   |-- CVC-300
|   |   |-- images
|   |   |-- masks
|   |-- CVC-ClinicDB
|   |   |-- images
|   |   |-- masks
|   |-- CVC-ColonDB
|   |   |-- images
|   |   |-- masks
|   |-- ETIS-LaribPolypDB
|   |   |-- images
|   |   |-- masks
|   |-- Kvasir
|       |-- images
|       |-- masks

Testing

  • Change the data_path in Test.py

Evaluation

  • Change the image_root and gt_root in eval_Kvasir.py
  • You can also run the matlab code in eval fold, it contains other four measurement metrics results.
  • You can download the segmentation maps of CaraNte from this link: CaraNet

Segmentation Results

  • Polyp Segmentation Results
Result
Result
  • Small polyp analysis

The x-axis is the proportion size (%) of polyp; y-axis is the average mean dice coefficient.

Kvasir CVC-ClinicDB CVC-ColonDB ETIS CVC-300
Result
Result
Result
Result
Result
  • Brain Tumor Segmentation Results
Result
  • Small tumor analysis
Result

Citation

@article{lou2021cfpnet,
  title={CFPNet: Channel-wise Feature Pyramid for Real-Time Semantic Segmentation},
  author={Lou, Ange and Loew, Murray},
  journal={arXiv preprint arXiv:2103.12212},
  year={2021}
}
Comments
  • About weighted intersection over union and weighted binary cross-entropy

    About weighted intersection over union and weighted binary cross-entropy

    Hi, thanks for your code! Can you explain to me why in structure_loss function use weights like this? weit = 1 + 5*torch.abs(F.avg_pool2d(mask, kernel_size=31, stride=1, padding=15) - mask)

    opened by maivan-hoa 6
  • dice_average.m and compute size of polyp

    dice_average.m and compute size of polyp

    Hi, I wonder to know how can I use the dice_average.m to compute the relationship between dice and the size of polyps. I'm not familiar with matalab , but dice_average.m seems only has a defined function ,I try to call this function in main.m files, but failed. Besides, I also curious about how did you compute the ratio of polyps , does the x axis 0,1,2,3,4,5,6 means the relative size ?

    opened by hybdxxw 6
  • About axial attention module gamma

    About axial attention module gamma

    Hello good sir,

    i am currently using your architecture as basis for my thesis work and while training the module i wondered about the axial attention module. you utilize a gamma value by setting it this way: self.gamma = nn.Parameter(torch.zeros(1)).

    Afterwards you multiply the result of´the axial attention with the gamma value: self.gamma * out and then apply the residual connection of x : out = self.gamma * out + x.

    Does this not mean you don't even utilize axial attention in any way by setting the output of axial attention to zero ? and only using the residual output ?

    Is this a different version of the code or am i missing something.

    I would be very thankful if you could elaborate on this.

    opened by SimonServant 5
  • About the main.m

    About the main.m

    Hi, sorry to bother. I'm wondering why I got all the Dice with NaN. I'v check the pathes, but still got NaN in each Dice. cell. The _result.txt is like "(Dataset:CVC-ClinicDB; Model:PraNet) meanDic:NaN;meanIoU:NaN;wFm:NaN;Sm:NaN;meanEm:NaN;MAE:NaN;maxEm:NaN;maxDice:NaN;maxIoU:NaN;meanSen:NaN;maxSen:NaN;meanSpe:NaN;maxSpe:NaN. "

    opened by 15104356116 5
  • do u do some ablation experments on ur modules

    do u do some ablation experments on ur modules

    Hi, thanks for ur great work, But I dont see ablation experiments whin ur net, like which part effective the performence more , Do u have done it? I'm very interesting in which modules matters most. thx

    opened by wcyjerry 3
  • Some questions about self_attn

    Some questions about self_attn

    Hi.

    1. Why there is no premute operation before view in mode h?
    # for mode h
    projected_query = self.query_conv(x).premute(0, 1, 3, 2).view(*view).permute(0, 2, 1)
    
    1. Why use sigmoid instead of softmax?
    opened by Asthestarsfalll 3
  • binary segmentation

    binary segmentation

    Hi, thank you for sharing your code. it is really helpful for me

    anyway, i have a question. I tried applying your code to my data. My data label values consist of 0,1 . When applied to this code, the output is 0-255. Can you tell me which part needs to be modified to adjust the output value to 0 and 1? I will wait for your reply. thank you

    opened by aram200 2
  • About the calculations of the model indicators  of every epoch

    About the calculations of the model indicators of every epoch

    Sorry to bother you. May I ask about how to calculate the IoU, precision, accuracy, recall , and DSC of every epoch? Or what variables should be used to calculate these indicators?I mean ,which variables are TN, FN, TP,FP.Sorry to trouble you!

    opened by 15104356116 2
  • About Axial-Attention

    About Axial-Attention

    Hi I notice that the derivation of axial-attention in the code is as follows: image

    But in the original version of axial attention it would look like this: image I don't fully understand your code, can you explain it for me?I am looking forward to hearing from u.

    opened by Liqq1 2
  • About Self-attn

    About Self-attn

    Regarding the use of self-attention. Before the activation function uses sigmoid, do you need to reduce d**-0.5, because Transformer uses softmax, and it is scaled before operation.

    opened by ZeriLinux 2
  • Why add a PRELU output to a linear output?

    Why add a PRELU output to a linear output?

    The output of the partial decoder (https://github.com/AngeLouCN/CaraNet/blob/main/lib/partial_decoder.py#L30) has a linear activation. The output of each of the axial attention modules, which are designed for residual learning, go through a BN-PReLU (https://github.com/AngeLouCN/CaraNet/blob/main/CaraNet.py#L47)

    The output (decoder + axial transformer 1, 2, 3, 4) then gets a sigmoid activation to generate class probabilities.

    Why modify the original linear output (from the partial decoder) with a nonlinear function that biases positive (PRELU)? Doesn't this mean that you're more likely to saturate the sigmoid by having a large input? Or at the very least result in exploding biases for the partial decoder?

    My understanding of residual learning is that it's commonly done with no activation functions prior to the summation to prevent exploding biases (continually adding a positive value to a positive value)

    opened by JohnMBrandt 1
  • mean Dice  is greater than 1 when I set --test_path to 'Kvasir' 'CVC-ColonDB' and 'ETIS-LaribPolypDB'

    mean Dice is greater than 1 when I set --test_path to 'Kvasir' 'CVC-ColonDB' and 'ETIS-LaribPolypDB'

    Excuse me, mean Dice is greater than 1 when I set --test_path to './TestDataset/Kvasir' './TestDataset/CVC-ColonDB' and './TestDataset/ETIS-LaribPolypDB'.I don't know why that is.

    opened by mingfuyan 1
Owner
GW (M.S. EE)→Vanderbilt (Ph.D. EE)
null
This is an official implementation for "AS-MLP: An Axial Shifted MLP Architecture for Vision".

AS-MLP architecture for Image Classification Model Zoo Image Classification on ImageNet-1K Network Resolution Top-1 (%) Params FLOPs Throughput (image

SVIP Lab 106 Dec 12, 2022
[MedIA2021]MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning

MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning [MedIA or Arxiv] and [Demo] This repository pr

Healthcare Intelligence Laboratory 92 Dec 8, 2022
Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms

LESA Introduction This repository contains the official implementation of Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Cont

Chenglin Yang 20 Dec 31, 2021
Official PyTorch implementation of UACANet: Uncertainty Aware Context Attention for Polyp Segmentation

UACANet: Uncertainty Aware Context Attention for Polyp Segmentation Official pytorch implementation of UACANet: Uncertainty Aware Context Attention fo

Taehun Kim 85 Dec 14, 2022
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

MIC-DKFZ 1.2k Jan 4, 2023
Build a medical knowledge graph based on Unified Language Medical System (UMLS)

UMLS-Graph Build a medical knowledge graph based on Unified Language Medical System (UMLS) Requisite Install MySQL Server 5.6 and import UMLS data int

Donghua Chen 6 Dec 25, 2022
Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation. Generally, MAS methods register multiple atlases, i.e., medical images with corresponding labels, to a target image;

NanYoMy 13 Oct 9, 2022
Adaptive Pyramid Context Network for Semantic Segmentation (APCNet CVPR'2019)

Adaptive Pyramid Context Network for Semantic Segmentation (APCNet CVPR'2019) Introduction Official implementation of Adaptive Pyramid Context Network

null 21 Nov 9, 2022
Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

Namt0d 9 Dec 7, 2022
Pytorch implementation of the paper "Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization"

Pytorch implementation of the paper "Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization"

Dongkyu Lee 4 Sep 18, 2022
A vision library for performing sliced inference on large images/small objects

SAHI: Slicing Aided Hyper Inference A vision library for performing sliced inference on large images/small objects Overview Object detection and insta

Open Business Software Solutions 2.3k Jan 4, 2023
Segcache: a memory-efficient and scalable in-memory key-value cache for small objects

Segcache: a memory-efficient and scalable in-memory key-value cache for small objects This repo contains the code of Segcache described in the followi

TheSys Group @ CMU CS 78 Jan 7, 2023
PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction... Attention in Attention Network for Image Super-Resolution (A2N) This repository is an PyTorch implementation of the paper "Atten

Haoyu Chen 71 Dec 30, 2022
codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification

DLCF-DCA codes for paper Combining Dynamic Local Context Focus and Dependency Cluster Attention for Aspect-level sentiment classification. submitted t

null 15 Aug 30, 2022
The code repository for "RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection" (ACM MM'21)

RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection (ACM MM'21) By Zhuofan Zong, Qianggang Cao, Biao Leng Introduction F

TempleX 9 Jul 30, 2022
Implementation of Segformer, Attention + MLP neural network for segmentation, in Pytorch

Segformer - Pytorch Implementation of Segformer, Attention + MLP neural network for segmentation, in Pytorch. Install $ pip install segformer-pytorch

Phil Wang 208 Dec 25, 2022
2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation

2021-MICCAI-Progressively Normalized Self-Attention Network for Video Polyp Segmentation Authors: Ge-Peng Ji*, Yu-Cheng Chou*, Deng-Ping Fan, Geng Che

Ge-Peng Ji (Daniel) 85 Dec 30, 2022
Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CVPR 2021)

Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CAC) Xin Lai*, Zhuotao Tian*, Li Jiang, Shu Liu, Hengshuang Zhao, Li

Jia Research Lab 137 Dec 14, 2022