Extension to fastai for volumetric medical data

Keno

Last update: Aug 22, 2022

Related tags

Deep Learning faimed3d

Overview

FAIMED 3D

use fastai to quickly train fully three-dimensional models on radiological data

Classification

from faimed3d.all import *

Load data in various medical formats (DICOM, NIfTI, NRRD) or even videos as simple as in fastai.

d = pd.read_csv('../data/radiopaedia_cases.csv')
dls = ImageDataLoaders3D.from_df(d,
                                 item_tfms = Resize3D((20, 112, 112)),
                                 batch_tfms = aug_transforms_3d(), 
                                 bs = 2, val_bs = 2)

Faimed3d provides multiple model architectures, pretrained on the UCF101 dataset for action recoginiton, which can be used for transfer learning.

Model	3-fold accuracy	duration/epoch	model size
efficientnet b0	92.5 %	9M:35S	48.8 MB
efficientnet b1	90.1 %	13M:20S	80.5 MB
resnet 18	87.6 %	6M:57S	339.1 MB
resnet 50	94.8 %	12M:16S	561.2 MB
resnet 101	96.0 %	17M:20S	1,030 MB

# slow
learn = cnn_learner_3d(dls, efficientnet_b0)

# slow
learn.lr_find()

SuggestedLRs(lr_min=0.014454397559165954, lr_steep=6.309573450380412e-07)

Click here for a more in-depth classification example.

Segmentation

dls = SegmentationDataLoaders3D.from_df(d,
                                 item_tfms = Resize3D((20, 112, 112)),
                                 batch_tfms = aug_transforms_3d(), 
                                 bs = 2, val_bs = 2)

All models in faimed3d can be used as a backbone for U-Nets, even with pre-trained weights.

# slow
learn = unet_learner_3d(dls, efficientnet_b0, n_out = 2)

# slow 
learn.lr_find()

SuggestedLRs(lr_min=0.33113112449646, lr_steep=0.10000000149011612)

Click here for a more in-depth segmentation example.

Comments

multiprocessing fails when reading DICOM series

Learner get stuck at the beginning of training if training images are loaded from a DICOM series. Training is still possible if num_workersargument in the dataloader is set to 0, but is very slow. Problem does not occur if training images are loaded from 3D NIfTI volumes.
bug

opened by kbressem 4
Viewer widget
Implementing a simple DICOM viewer as iPython Widget.

Mentioned in #20

Motivation

Currently, the 3D volumes were displayed as multiple tiles in a mosaic style. But this is not how a radiologist looks at images. Instead, radiologists view images as a scrollable stack of 2D slices.

Solution

iPywidgets allows rendering one stack of image slices into a scrollable widget.

With the left lower slider, one can scroll through the image stack; with the upper slider, it is possible to apply windowing to the images. If a segmentation mask is passed together with the image, it can be overlayed to the image. With the lower right button, the mask overlay can be toggled on or off.

This simple DICOM viewer is now used as a replacement for the show_batch_3d function.

Additionally, another viewer version gives more details on the pixel/voxel distribution in the image. This could be useful for visualizing DICOM images while preprocessing (normalizing, bias correction).

Files changed

added 09_widgets.viewer to notebooks

changed show_batch_3d in 04_dataloaders

cleaned dev_nbs and nbs folder

some small style changes
opened by kbressem 2
Fix 4d unet

fix for issue #25

Changed architecture of 4D UNet, to enable efficient use of the hooks.

Encoder consists now out of two submodules: MultiStem and RepeatedSequential. In MultiStem the encoder stem is repeated but weights are not shared. For each 3D element in the 4D batch a uniqe stem is used. RepeatedSequential passes multiple items in a batch through the same network. It expects a list of tensors and returns a list of tensors. Hooks will not return a list of tensors after each RepeatedSequential which can be concatenated in the UNet Block.

Overall, this approach allows to use multiple 3D inputs for segmentation and keeps the feature maps for each input for the upscaling blocks.

opened by kbressem 2
In-depth example links leads to error 404
Describe the bug in-depth examples link leads to error 404 To Reproduce Steps to reproduce the behavior:

Go to https://github.com/kbressem/faimed3d/blob/main/README.md

Click here for a more in-depth classification example.

or here for a more in-depth segmentation example.

See error

Screenshots

Additional context Add any other context about the problem here.
opened by omer-dor 1
Bump addressable from 2.7.0 to 2.8.0 in /docs
Bumps addressable from 2.7.0 to 2.8.0.

Changelog

Sourced from addressable's changelog.

Addressable 2.8.0

fixes ReDoS vulnerability in Addressable::Template#match

no longer replaces + with spaces in queries for non-http(s) schemes

fixed encoding ipv6 literals

the :compacted flag for normalized_query now dedupes parameters

fix broken escape_component alias

dropping support for Ruby 2.0 and 2.1

adding Ruby 3.0 compatibility for development tasks

drop support for rack-mount and remove Addressable::Template#generate

performance improvements

switch CI/CD to GitHub Actions

Commits

6469a23 Updating gemspec again

2433638 Merge branch 'main' of github.com:sporkmonger/addressable into main

e9c76b8 Merge pull request #378 from ashmaroli/flat-map

56c5cf7 Update the gemspec

c1fed1c Require a non-vulnerable rake

0d8a312 Adding note about ReDoS vulnerability

89c7613 Merge branch 'template-regexp' into main

cf8884f Note about alias fix

bb03f71 Merge pull request #371 from charleystran/add_missing_encode_component_doc_entry

6d1d809 Adding note about :compacted normalization

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
learn.unfreeze() does not work if opt_state is saved

Describe the bug If a 3D CNN model is saved with an optimizer state, unfreezing the model will fail.

Expected behavior learn.unfreeze() should work for all models, regardless if the optimizer state was saved.
bug

opened by kbressem 1

Multiprocessing fails for ImageDataLoaders with multiple inputs

Describe the bug When using multiple inputs in a CNN, the Learner gets stuck at the start.

To Reproduce Steps to reproduce the behavior:

d = pd.read_csv('../data/radiopaedia_cases.csv')
dls = ImageDataLoaders3D.from_df(d,
                                 fn_col = [0,1],
                                size_for_resampling = (112, 112, 20),
                                 label_col = 2, # in sample data 0: series, 1: segmentation mask, 2: random binary label
                                 bs = 2, 
                                 val_bs = 2
                                 )

learn = cnn_learner_3d(dls, r3d_18, pretrained=False)

learn.fine_tune(1, 0.001)

Screenshots Traceback shows that the model gets stuck at the dataloader,

Workaround until issue is fixed is to set num_workers=0, but this makes the training longer.

bug

opened by kbressem 1

Bump addressable from 2.7.0 to 2.8.1 in /docs
Bumps addressable from 2.7.0 to 2.8.1.

Changelog

Sourced from addressable's changelog.

Addressable 2.8.1

refactor Addressable::URI.normalize_path to address linter offenses (#430)

remove redundant colon in Addressable::URI::CharacterClasses::AUTHORITY regex (#438)

update gemspec to reflect supported Ruby versions (#466, #464, #463)

compatibility w/ public_suffix 5.x (#466, #465, #460)

fixes "invalid byte sequence in UTF-8" exception when unencoding URLs containing non UTF-8 characters (#459)

Ractor compatibility (#449)

use the whole string instead of a single line for template match (#431)

force UTF-8 encoding only if needed (#341)

#460: sporkmonger/addressable#460 #463: sporkmonger/addressable#463 #464: sporkmonger/addressable#464 #465: sporkmonger/addressable#465 #466: sporkmonger/addressable#466

Addressable 2.8.0

fixes ReDoS vulnerability in Addressable::Template#match

no longer replaces + with spaces in queries for non-http(s) schemes

fixed encoding ipv6 literals

the :compacted flag for normalized_query now dedupes parameters

fix broken escape_component alias

dropping support for Ruby 2.0 and 2.1

adding Ruby 3.0 compatibility for development tasks

drop support for rack-mount and remove Addressable::Template#generate

performance improvements

switch CI/CD to GitHub Actions

Commits

8657465 Update version, gemspec, and CHANGELOG for 2.8.1 (#474)

4fc5bb6 CI: remove Ubuntu 18.04 job (#473)

860fede Force UTF-8 encoding only if needed (#341)

99810af Merge pull request #431 from ojab/ct-_do_not_parse_multiline_strings

7ce0f48 Merge branch 'main' into ct-_do_not_parse_multiline_strings

7ecf751 Merge pull request #449 from okeeblow/freeze_concatenated_strings

41f12dd Merge branch 'main' into freeze_concatenated_strings

068f673 Merge pull request #459 from jarthod/iso-encoding-problem

b4c9882 Merge branch 'main' into iso-encoding-problem

08d27e8 Merge pull request #471 from sporkmonger/sporkmonger-enable-codeql

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump tzinfo from 1.2.9 to 1.2.10 in /docs
Bumps tzinfo from 1.2.9 to 1.2.10.

Release notes

Sourced from tzinfo's releases.

v1.2.10

Fixed a relative path traversal bug that could cause arbitrary files to be loaded with require when used with RubyDataSource. Please refer to https://github.com/tzinfo/tzinfo/security/advisories/GHSA-5cm2-9h8c-rvfx for details. CVE-2022-31163.

Ignore the SECURITY file from Arch Linux's tzdata package. #134.

TZInfo v1.2.10 on RubyGems.org

Changelog

Sourced from tzinfo's changelog.

Version 1.2.10 - 19-Jul-2022

Fixed a relative path traversal bug that could cause arbitrary files to be loaded with require when used with RubyDataSource. Please refer to https://github.com/tzinfo/tzinfo/security/advisories/GHSA-5cm2-9h8c-rvfx for details. CVE-2022-31163.

Ignore the SECURITY file from Arch Linux's tzdata package. #134.

Commits

0814dcd Fix the release date.

fd05e2a Preparing v1.2.10.

b98c32e Merge branch 'fix-directory-traversal-1.2' into 1.2

ac3ee68 Remove unnecessary escaping of + within regex character classes.

9d49bf9 Fix relative path loading tests.

394c381 Remove private_constant for consistency and compatibility.

5e9f990 Exclude Arch Linux's SECURITY file from the time zone index.

17fc9e1 Workaround for 'Permission denied - NUL' errors with JRuby on Windows.

6bd7a51 Update copyright years.

9905ca9 Fix directory traversal in Timezone.get when using Ruby data source

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0

Owner

Keno

Assistant professor of radiology interested in deep learning

GitHub

A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

A PyTorch implementation of V-Net Vnet is a PyTorch implementation of the paper V-Net: Fully Convolutional Neural Networks for Volumetric Medical Imag

606 Dec 21, 2022

Pytorch Code for "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation"

Medical-Transformer Pytorch Code for the paper "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation" About this repo: This repo

615 Dec 25, 2022

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

1.2k Jan 4, 2023

Build a medical knowledge graph based on Unified Language Medical System (UMLS)

UMLS-Graph Build a medical knowledge graph based on Unified Language Medical System (UMLS) Requisite Install MySQL Server 5.6 and import UMLS data int

6 Dec 25, 2022

This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"

697 Jan 6, 2023

Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Point-Unet This is an unofficial implementation of the MICCAI 2021 paper Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segment

9 Dec 7, 2022

Code for A Volumetric Transformer for Accurate 3D Tumor Segmentation

VT-UNet This repo contains the supported pytorch code and configuration files to reproduce 3D medical image segmentaion results of VT-UNet. Environmen

7 Dec 3, 2021

The fastai deep learning library

Welcome to fastai fastai simplifies training fast and accurate neural nets using modern best practices Important: This documentation covers fastai v2,

23.2k Jan 7, 2023

The fastai deep learning library

Welcome to fastai fastai simplifies training fast and accurate neural nets using modern best practices Important: This documentation covers fastai v2,

20.4k Feb 12, 2021

tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series classification, regression and forecasting.

Time series Timeseries Deep Learning Pytorch fastai - State-of-the-art Deep Learning with Time Series and Sequences in Pytorch / fastai

2.8k Jan 8, 2023

fastgradio is a python library to quickly build and share gradio interfaces of your trained fastai models.

34 Jan 5, 2023

An Agnostic Computer Vision Framework - Pluggable to any Training Library: Fastai, Pytorch-Lightning with more to come

IceVision is the first agnostic computer vision framework to offer a curated collection with hundreds of high-quality pre-trained models from torchvision, MMLabs, and soon Pytorch Image Models. It orchestrates the end-to-end deep learning workflow allowing to train networks with easy-to-use robust high-performance libraries such as Pytorch-Lightning and Fastai

789 Dec 29, 2022

Official and maintained implementation of the paper "OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data" [BMVC 2021].

OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data Christoph Reich, Tim Prangemeier, Özdemir Cetin & Heinz Koeppl | Pr

23 Sep 21, 2022

A library of extension and helper modules for Python's data analysis and machine learning libraries.

Mlxtend (machine learning extensions) is a Python library of useful tools for the day-to-day data science tasks. Sebastian Raschka 2014-2020 Links Doc

4.2k Jan 2, 2023

A scanpy extension to analyse single-cell TCR and BCR data.

Scirpy: A Scanpy extension for analyzing single-cell immune-cell receptor sequencing data Scirpy is a scalable python-toolkit to analyse T cell recept

145 Jan 3, 2023

Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data based on Pytorch Framework

VFedPCA+VFedAKPCA This is the official source code for the Paper: Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-

9 Sep 18, 2022

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

49 Nov 23, 2022

This repository is an unoffical PyTorch implementation of Medical segmentation in 3D and 2D.

Pytorch Medical Segmentation Read Chinese Introduction：Here！ Recent Updates 2021.1.8 The train and test codes are released. 2021.2.6 A bug in dice was

618 Dec 27, 2022

This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

TransUNet This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation Usage

1.4k Jan 4, 2023

Extension to fastai for volumetric medical data

Related tags

Overview

FAIMED 3D

Classification

Segmentation

Comments

Implementing a simple DICOM viewer as iPython Widget.

Motivation

Solution

Files changed

Addressable 2.8.0

Addressable 2.8.1

Addressable 2.8.0

v1.2.10

Version 1.2.10 - 19-Jul-2022

Owner

Keno

A PyTorch implementation for V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation

Pytorch Code for "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation"

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

Build a medical knowledge graph based on Unified Language Medical System (UMLS)

This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"

Unofficial implementation of Point-Unet: A Context-Aware Point-Based Neural Network for Volumetric Segmentation

Code for A Volumetric Transformer for Accurate 3D Tumor Segmentation

The fastai deep learning library

The fastai deep learning library

tsai is an open-source deep learning package built on top of Pytorch & fastai focused on state-of-the-art techniques for time series classification, regression and forecasting.

fastgradio is a python library to quickly build and share gradio interfaces of your trained fastai models.

An Agnostic Computer Vision Framework - Pluggable to any Training Library: Fastai, Pytorch-Lightning with more to come

Official and maintained implementation of the paper "OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data" [BMVC 2021].

A library of extension and helper modules for Python's data analysis and machine learning libraries.

A scanpy extension to analyse single-cell TCR and BCR data.

Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data based on Pytorch Framework

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

This repository is an unoffical PyTorch implementation of Medical segmentation in 3D and 2D.

This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation