Medical image analysis framework merging ANTsPy and deep learning

Last update: Dec 24, 2022

Overview

ANTsPyNet

A collection of deep learning architectures and applications ported to the python language and tools for basic medical image processing. Based on keras and tensorflow with cross-compatibility with our R analog ANTsRNet.

Documentation page https://antsx.github.io/ANTsPyNet/.

For MacOS and Linux, may install with:

pip install antspynet

Architectures

Image voxelwise segmentation/regression

Image classification/regression

Object detection

Image super-resolution

Registration and transforms

Spatial transformer network (STN) (2-D, 3-D)

Generative adverserial networks

Clustering

Applications

MRI super-resolution
Multi-modal brain extraction
- T1
- T1 "no brainer"
- FLAIR
- T2
- FA
- BOLD
- T1/T2 infant
Six-tissue Atropos brain segmentation
Cortical thickness
Brain age
Hippmapp3r hippocampal segmentation
White matter hyperintensity segmentation
Hypothalamus segmentation
Claustrum segmentation
Deep Flash
Desikan-Killiany-Tourville cortical labeling
Lung extraction
- Proton
- CT
Functional lung segmentation
Neural style transfer
Image quality assessment
- TID2013
- KonIQ-10k

Miscellaneous

Mixture density networks (MDN)

Installation

ANTsPyNet Installation:

Option 1:

$ git clone https://github.com/ANTsX/ANTsPyNet
$ cd ANTsPyNet
$ python setup.py install

Publications

Nicholas J. Tustison, Talissa A. Altes, Kun Qing, Mu He, G. Wilson Miller, Brian B. Avants, Yun M. Shim, James C. Gee, John P. Mugler III, and Jaime F. Mata. Image- versus histogram-based considerations in semantic segmentation of pulmonary hyperpolarized gas images. Magnetic Resonance in Medicine. (pubmed)
Nicholas J. Tustison, Philip A. Cook, Andrew J. Holbrook, Hans J. Johnson, John Muschelli, Gabriel A. Devenyi, Jeffrey T. Duda, Sandhitsu R. Das, Nicholas C. Cullen, Daniel L. Gillen, Michael A. Yassa, James R. Stone, James C. Gee, and Brian B. Avants for the Alzheimer’s Disease Neuroimaging Initiative. The ANTsX ecosystem for quantitative biological and medical imaging. Scientific Reports. 11(1):9068, Apr 2021. (pubmed)
Andrew T. Grainger, Arun Krishnaraj, Michael H. Quinones, Nicholas J. Tustison, Samantha Epstein, Daniela Fuller, Aakash Jha, Kevin L. Allman, Weibin Shi. Deep Learning-based Quantification of Abdominal Subcutaneous and Visceral Fat Volume on CT Images, Academic Radiology. (pubmed)
Nicholas J. Tustison, Brian B. Avants, and James C. Gee. Learning image-based spatial transformations via convolutional neural networks: a review, Magnetic Resonance Imaging, 64:142-153, Dec 2019. (pubmed)
Nicholas J. Tustison, Brian B. Avants, Zixuan Lin, Xue Feng, Nicholas Cullen, Jaime F. Mata, Lucia Flors, James C. Gee, Talissa A. Altes, John P. Mugler III, and Kun Qing. Convolutional Neural Networks with Template-Based Data Augmentation for Functional Lung Image Quantification, Academic Radiology, 26(3):412-423, Mar 2019. (pubmed)
Andrew T. Grainger, Nicholas J. Tustison, Kun Qing, Rene Roy, Stuart S. Berr, and Weibin Shi. Deep learning-based quantification of abdominal fat on magnetic resonance images. PLoS One, 13(9):e0204071, Sep 2018. (pubmed)
Cullen N.C., Avants B.B. (2018) Convolutional Neural Networks for Rapid and Simultaneous Brain Extraction and Tissue Segmentation. In: Spalletta G., Piras F., Gili T. (eds) Brain Morphometry. Neuromethods, vol 136. Humana Press, New York, NY doi

Acknowledgments

We gratefully acknowledge the support of the NVIDIA Corporation with the donation of two Titan Xp GPUs used for this research.
We gratefully acknowledge the grant support of the Office of Naval Research and Cohen Veterans Bioscience.

Comments

Inconsistent patch size from antspynet.extract_image_patches

hi @ntustison - do you know a quick solution to this?

img=ants.image_read( ants.get_data( "r16" ) )
radder=[20,20]
msk=ants.get_mask( img )
ptch = antspynet.extract_image_patches( img, tuple(radder), mask_image=msk, 
  max_number_of_patches = 512, return_as_array=False )
for k in range(512):
     print( ptch[k].shape )

shows patches of variable sizes. when trying to compute consistent patch statistics, this can lead to issues of power. I believe this behavior is different from antsrnet.

img=ri(1)
radder=c(20,20)
msk=getMask( img )
ptch = extractImagePatches( img, radder, maskImage=msk, 
  max_number_of_patches = 512, return_as_array=False )

opened by stnava 13

Error on tf.keras.models.load_model, missing 'trainable' argument
Similar to #50, I get this error whenever trying to load a model:

>>> super = antspynet.mri_super_resolution(anat, verbose = True) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/pcook/tmp/NOT_BACKED_UP/antsPyNetDevel/ANTsPyNet/antspynet/utilities/mri_super_resolution.py", line 45, in mri_super_resolution model_sr = tf.keras.models.load_model(model_and_weights_file_name, compile=False) File "/opt/miniconda3/envs/antspynetdevel/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/opt/miniconda3/envs/antspynetdevel/lib/python3.8/site-packages/keras/engine/base_layer.py", line 340, in __init__ raise TypeError( TypeError: Expected `trainable` argument to be a boolean, but got: None I got the same error with a different model

>>> model = tf.keras.models.load_model(antspynet.get_pretrained_network("koniqMS"), compile=False) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/opt/miniconda3/envs/antspynetdevel/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/opt/miniconda3/envs/antspynetdevel/lib/python3.8/site-packages/keras/engine/base_layer.py", line 340, in __init__ raise TypeError( TypeError: Expected `trainable` argument to be a boolean, but got: None

The workaround in #50 was to edit the keras base_layer.py code, is there another solution? This is with the current master antspynet, on my Mac Catalina. Here's my list of installed packages in this environment

% pip freeze absl-py==1.0.0 -e git+ssh://[email protected]/ANTsX/ANTsPyNet.git@011d3e22e69ed8e9668b077868f695c115c0138f#egg=antspynet antspyx==0.3.2 astunparse==1.6.3 cachetools==5.1.0 certifi==2022.5.18.1 charset-normalizer==2.0.12 chart-studio==1.1.0 cloudpickle==2.1.0 cycler==0.11.0 decorator==5.1.1 dm-tree==0.1.7 flatbuffers==1.12 fonttools==4.33.3 gast==0.4.0 google-auth==2.6.6 google-auth-oauthlib==0.4.6 google-pasta==0.2.0 grpcio==1.46.3 h5py==3.7.0 idna==3.3 imageio==2.19.2 importlib-metadata==4.11.4 joblib==1.1.0 keras==2.9.0 Keras-Preprocessing==1.1.2 kiwisolver==1.4.2 libclang==14.0.1 Markdown==3.3.7 matplotlib==3.5.2 networkx==2.8.2 nibabel==3.2.2 numpy==1.22.4 oauthlib==3.2.0 opt-einsum==3.3.0 packaging==21.3 pandas==1.4.2 patsy==0.5.2 Pillow==9.1.1 plotly==5.8.0 protobuf==3.19.4 pyasn1==0.4.8 pyasn1-modules==0.2.8 pyparsing==3.0.9 python-dateutil==2.8.2 pytz==2022.1 PyWavelets==1.3.0 PyYAML==6.0 requests==2.27.1 requests-oauthlib==1.3.1 retrying==1.3.3 rsa==4.8 scikit-image==0.19.2 scikit-learn==1.1.1 scipy==1.8.1 six==1.16.0 statsmodels==0.13.2 tenacity==8.0.1 tensorboard==2.9.0 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.1 tensorflow==2.9.1 tensorflow-estimator==2.9.0 tensorflow-io-gcs-filesystem==0.26.0 tensorflow-probability==0.16.0 termcolor==1.1.0 threadpoolctl==3.1.0 tifffile==2022.5.4 typing_extensions==4.2.0 urllib3==1.26.9 webcolors==1.11.1 Werkzeug==2.1.2 wrapt==1.14.1 zipp==3.8.0
opened by cookpa 11

Resampling Utilities Example

xray

just wondering how this should work on the above image.

try 1:

import ants
img = ants.image_read("xray.jpg").split_channels()
for k in range(3):
    img[k] = ants.crop_indices( img[k], (0,0),(1926,2204) ) # dyadic

import antspynet
import tensorflow as tf
import keras as k
# set up the resampling using tf
input = k.Input( img[0].numpy().shape )
outdim = ( 1926/2, 2204/2 )
up = antspynet.ResampleTensorLayer2D( outdim, 'linear', name='xxx' )(input)

any working examples around?

opened by stnava 11

FIX: to neural assessment and BXT

also - need to update the assessment models as they have None instead of False


mdl=tf.keras.models.load_model(mfn, compile=False)
for k in range( len(mdl.layers) ):
  if mdl.layers[k].trainable is None:
    mdl.layers[k].trainable=False
tf.keras.models.save_model( mdl, mfn )

will send those models later

the bxt fix just thresholds the sigmoid function.

opened by stnava 7

Figshare download error on pretrained network

Hi @ntustison,

I am getting a 403 error for this network, it looks like the URL changed in May, but maybe it needs to be made public? https://github.com/ANTsX/ANTsPyNet/blob/42a98b81e1efd2767152bf867cd588a02338e1ec/antspynet/utilities/get_pretrained_network.py#L66

opened by cookpa 6
ANTsPyNet installation error: Tensorflow not found

Hi Everyone,

I'm trying to install ANTsPyNet using the instructions in https://github.com/ANTsX/ANTsPyNet.

Everything seemed to go well (please see the attached log), except that the final command generates a "tensorflow distribution not found" error as follows:

jay@Euler:~/ANTsPyNet$ sudo python3 setup.py install ... ... Installed /usr/local/lib/python3.8/dist-packages/tfp_nightly-0.2.0.dev20220302-py3.8.egg error: The 'tensorflow' distribution was not found and is required by antspynet jay@Euler:~/ANTsPyNet$

Can you please help? Thank you! Jay

ANTsPyNet_installation_log_JayHegde.txt

opened by hegde00 6

Conflicting dependencies (Apple Silicon M1 hardware Monterey 12.0.1)

Machine: MacBook Pro M1 2020 OS: macOS Monterey 12.0.1 Python version of venv: Python 3.8.12 Pip version: 21.3.1 I have installed Tensorflow using this guide.

When I execute pip install antspynet I get the following error:

Collecting antspynet
Using cached antspynet-0.1.3-py3-none-any.whl (151 kB)

Collecting tensorflow-probability
Using cached tensorflow_probability-0.15.0-py2.py3-none-any.whl (5.7 MB)

Collecting antspyx
Using cached antspyx-0.2.9.tar.gz (305.4 MB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: statsmodels in /Users/htrivino/miniforge3/lib/python3.8/site-packages (from antspynet) (0.12.1)
Requirement already satisfied: requests in /Users/htrivino/miniforge3/lib/python3.8/site-packages (from antspynet) (2.26.0)
Requirement already satisfied: scikit-learn in /Users/htrivino/miniforge3/lib/python3.8/site-packages (from antspynet) (1.0.1)

Collecting antspynet
Using cached antspynet-0.1.2-py3-none-any.whl (143 kB)
Using cached antspynet-0.1.1-py3-none-any.whl (140 kB)
ERROR: Cannot install antspynet==0.1.1, antspynet==0.1.2 and antspynet==0.1.3 because these package versions have conflicting dependencies.

The conflict is caused by:
antspynet 0.1.3 depends on tensorflow
antspynet 0.1.2 depends on tensorflow
antspynet 0.1.1 depends on tensorflow

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

This is the conda list of the environment.

# packages in environment at /Users/htrivino/miniforge3:
#
# Name                    Version                   Build  Channel
absl-py                   0.10.0             pyhd8ed1ab_1    conda-forge
aiohttp                   3.7.4.post0      py38hea4295b_1    conda-forge
ants                      0.0.7                    pypi_0    pypi
appnope                   0.1.2            py38h10201cd_2    conda-forge
argon2-cffi               21.1.0           py38hea4295b_2    conda-forge
asgiref                   3.4.1                    pypi_0    pypi
astor                     0.8.1              pyh9f0ad1d_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
async-timeout             3.0.1                   py_1000    conda-forge
async_generator           1.10                       py_0    conda-forge
attrs                     21.2.0             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                        py_2    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
bleach                    4.1.0              pyhd8ed1ab_0    conda-forge
blinker                   1.4                        py_1    conda-forge
bokeh                     2.3.3            py38h10201cd_0    conda-forge
brotlipy                  0.7.0           py38hea4295b_1003    conda-forge
c-ares                    1.18.1               h3422bc3_0    conda-forge
ca-certificates           2021.10.8            h4653dfc_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                4.2.4              pyhd8ed1ab_0    conda-forge
certifi                   2021.10.8        py38h10201cd_1    conda-forge
cffi                      1.15.0           py38hc67bbb8_0    conda-forge
chardet                   4.0.0            py38h10201cd_2    conda-forge
charset-normalizer        2.0.0              pyhd8ed1ab_0    conda-forge
clang                     5.0                      pypi_0    pypi
click                     8.0.3            py38h10201cd_1    conda-forge
cloudpickle               2.0.0              pyhd8ed1ab_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
conda                     4.10.3           py38h10201cd_3    conda-forge
conda-package-handling    1.7.3            py38hea4295b_1    conda-forge
cryptography              35.0.0           py38h10d4710_2    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cytoolz                   0.11.2           py38hea4295b_1    conda-forge
dask                      2021.11.1          pyhd8ed1ab_0    conda-forge
dask-core                 2021.11.1          pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
decorator                 5.1.0              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
distributed               2021.11.1        py38h10201cd_0    conda-forge
django                    3.2.9                    pypi_0    pypi
entrypoints               0.3             pyhd8ed1ab_1003    conda-forge
flatbuffers               1.12                     pypi_0    pypi
freetype                  2.10.4               h17b34a0_1    conda-forge
fsspec                    2021.11.0          pyhd8ed1ab_0    conda-forge
gast                      0.4.0              pyh9f0ad1d_0    conda-forge
gevent                    21.8.0                   pypi_0    pypi
google-auth               1.35.0             pyh6c4a22f_0    conda-forge
google-auth-oauthlib      0.4.6              pyhd8ed1ab_0    conda-forge
google-pasta              0.2.0              pyh8c360ce_0    conda-forge
greenlet                  1.1.2                    pypi_0    pypi
grpcio                    1.41.1           py38h69ee544_1    conda-forge
h5py                      3.1.0           nompi_py38h032b01a_100    conda-forge
hdf5                      1.10.6          nompi_h0fc092c_1114    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
idna                      3.1                pyhd3deb0d_0    conda-forge
importlib-metadata        4.8.2            py38h10201cd_0    conda-forge
importlib_resources       5.4.0              pyhd8ed1ab_0    conda-forge
ipykernel                 5.5.5            py38h2cb4d76_0    conda-forge
ipython                   7.29.0           py38h2cb4d76_1    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                7.6.5              pyhd8ed1ab_0    conda-forge
jbig                      2.1               h3422bc3_2003    conda-forge
jedi                      0.18.0           py38h10201cd_3    conda-forge
jinja2                    3.0.3              pyhd8ed1ab_0    conda-forge
joblib                    1.1.0              pyhd8ed1ab_0    conda-forge
jpeg                      9d                   h27ca646_0    conda-forge
jsonschema                4.2.1              pyhd8ed1ab_0    conda-forge
jupyter                   1.0.0            py38h10201cd_6    conda-forge
jupyter_client            7.0.6              pyhd8ed1ab_0    conda-forge
jupyter_console           6.4.0              pyhd8ed1ab_0    conda-forge
jupyter_core              4.9.1            py38h10201cd_0    conda-forge
jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
jupyterlab_widgets        1.0.2              pyhd8ed1ab_0    conda-forge
keras                     2.6.0              pyhd8ed1ab_0    conda-forge
keras-preprocessing       1.1.2              pyhd8ed1ab_0    conda-forge
kiwisolver                1.3.2            py38h1670459_1    conda-forge
krb5                      1.19.2               hd92b7a7_3    conda-forge
lcms2                     2.12                 had6a04f_0    conda-forge
lerc                      3.0                  hbdafb3b_0    conda-forge
libblas                   3.9.0           12_osxarm64_openblas    conda-forge
libcblas                  3.9.0           12_osxarm64_openblas    conda-forge
libclang                  11.1.0          default_h0fdd720_1    conda-forge
libcurl                   7.80.0               h8fe1914_0    conda-forge
libcxx                    12.0.1               h168391b_0    conda-forge
libdeflate                1.8                  h3422bc3_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h642e427_1    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0.dev0      11_0_1_hf114ba7_23    conda-forge
libgfortran5              11.0.1.dev0         hf114ba7_23    conda-forge
liblapack                 3.9.0           12_osxarm64_openblas    conda-forge
libllvm11                 11.1.0               h93073aa_2    conda-forge
libnghttp2                1.43.0               he4cd7f6_1    conda-forge
libopenblas               0.3.18          openmp_h5dd58f0_0    conda-forge
libpng                    1.6.37               hf7e6567_2    conda-forge
libprotobuf               3.19.1               hccf11d3_0    conda-forge
libsodium                 1.0.18               h27ca646_1    conda-forge
libssh2                   1.10.0               hb80f160_2    conda-forge
libtiff                   4.3.0                h74060c4_2    conda-forge
libwebp-base              1.2.1                h3422bc3_0    conda-forge
libzlib                   1.2.11            hee7b306_1013    conda-forge
llvm-openmp               12.0.1               hf3c4609_1    conda-forge
locket                    0.2.0                      py_2    conda-forge
lz4-c                     1.9.3                hbdafb3b_1    conda-forge
markdown                  3.3.4              pyhd8ed1ab_0    conda-forge
markupsafe                2.0.1            py38hea4295b_1    conda-forge
matplotlib-base           3.4.3            py38hb140015_1    conda-forge
matplotlib-inline         0.1.3              pyhd8ed1ab_0    conda-forge
mistune                   0.8.4           py38hea4295b_1005    conda-forge
msgpack-python            1.0.2            py38h1670459_2    conda-forge
multidict                 5.2.0            py38hea4295b_1    conda-forge
nb_conda_kernels          2.3.1            py38h10201cd_0    conda-forge
nbclient                  0.5.5              pyhd8ed1ab_0    conda-forge
nbconvert                 6.2.0            py38h10201cd_0    conda-forge
nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
ncurses                   6.2                  h9aa5885_4    conda-forge
nest-asyncio              1.5.1              pyhd8ed1ab_0    conda-forge
notebook                  6.4.5              pyha770c72_0    conda-forge
numpy                     1.19.5           py38hbf7bb01_2    conda-forge
oauthlib                  3.1.1              pyhd8ed1ab_0    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openjpeg                  2.4.0                h062765e_1    conda-forge
openssl                   1.1.1l               h3422bc3_0    conda-forge
opt_einsum                3.3.0              pyhd8ed1ab_1    conda-forge
packaging                 21.0               pyhd8ed1ab_0    conda-forge
pandas                    1.3.4            py38h3777fb4_1    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
parso                     0.8.2              pyhd8ed1ab_0    conda-forge
partd                     1.2.0              pyhd8ed1ab_0    conda-forge
patsy                     0.5.2              pyhd8ed1ab_0    conda-forge
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    8.4.0            py38h02acf36_0    conda-forge
pip                       21.3.1                   pypi_0    pypi
plotly                    5.3.1              pyhd8ed1ab_0    conda-forge
prometheus_client         0.12.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.22             pyha770c72_0    conda-forge
prompt_toolkit            3.0.22               hd8ed1ab_0    conda-forge
protobuf                  3.19.1           py38h6f2b01f_1    conda-forge
psutil                    5.8.0            py38hea4295b_2    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pyasn1                    0.4.8                      py_0    conda-forge
pyasn1-modules            0.2.7                      py_0    conda-forge
pycosat                   0.6.3           py38hea4295b_1009    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pygments                  2.10.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.3.0              pyhd8ed1ab_0    conda-forge
pyopenssl                 21.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.5              pyhd8ed1ab_0    conda-forge
pyrsistent                0.18.0           py38hea4295b_0    conda-forge
pysocks                   1.7.1            py38h10201cd_4    conda-forge
python                    3.8.12          hab31e5c_2_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.8                      2_cp38    conda-forge
pytz                      2021.3             pyhd8ed1ab_0    conda-forge
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
pyyaml                    6.0              py38hea4295b_3    conda-forge
pyzmq                     22.3.0           py38h51b17a6_1    conda-forge
readline                  8.1                  hedafd6a_0    conda-forge
requests                  2.26.0             pyhd8ed1ab_0    conda-forge
requests-oauthlib         1.3.0              pyh9f0ad1d_0    conda-forge
rsa                       4.7.2              pyh44b312d_0    conda-forge
ruamel_yaml               0.15.80         py38hea4295b_1006    conda-forge
scikit-learn              1.0.1            py38hfeda2c9_1    conda-forge
scipy                     1.7.2            py38hd0c9ec0_0    conda-forge
seaborn                   0.11.2               hd8ed1ab_0    conda-forge
seaborn-base              0.11.2             pyhd8ed1ab_0    conda-forge
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
setuptools                58.5.3           py38h10201cd_0    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
sqlite                    3.36.0               h72a2b83_2    conda-forge
sqlparse                  0.4.2                    pypi_0    pypi
statsmodels               0.12.1           py38h8369297_2    conda-forge
tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
tenacity                  8.0.1              pyhd8ed1ab_0    conda-forge
tensorboard               2.6.0              pyhd8ed1ab_1    conda-forge
tensorboard-data-server   0.6.0            py38h10d4710_1    conda-forge
tensorboard-plugin-wit    1.8.0              pyh44b312d_0    conda-forge
tensorflow-deps           2.6.0                         0    apple
tensorflow-estimator      2.6.0            py38hddd8853_0    conda-forge
tensorflow-macos          2.6.0                    pypi_0    pypi
tensorflow-metal          0.2.0                    pypi_0    pypi
termcolor                 1.1.0                      py_2    conda-forge
terminado                 0.12.1           py38h10201cd_1    conda-forge
testpath                  0.5.0              pyhd8ed1ab_0    conda-forge
threadpoolctl             3.0.0              pyh8a188c0_0    conda-forge
tk                        8.6.11               he1e0b03_1    conda-forge
toolz                     0.11.2             pyhd8ed1ab_0    conda-forge
tornado                   6.1              py38hea4295b_2    conda-forge
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
traitlets                 5.1.1              pyhd8ed1ab_0    conda-forge
tsfresh                   0.17.0                     py_0    conda-forge
typeguard                 2.13.0             pyhd8ed1ab_0    conda-forge
typing-extensions         3.7.4.3                       0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
urllib3                   1.26.7             pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
webencodings              0.5.1                      py_1    conda-forge
werkzeug                  2.0.1              pyhd8ed1ab_0    conda-forge
wheel                     0.35.1             pyh9f0ad1d_0    conda-forge
widgetsnbextension        3.5.2            py38h10201cd_0    conda-forge
wrapt                     1.12.1           py38hea4295b_3    conda-forge
xz                        5.2.5                h642e427_1    conda-forge
yaml                      0.2.5                h642e427_0    conda-forge
yarl                      1.7.2            py38hea4295b_1    conda-forge
zeromq                    4.3.4                hbdafb3b_1    conda-forge
zict                      2.0.0                      py_0    conda-forge
zipp                      3.6.0              pyhd8ed1ab_0    conda-forge
zlib                      1.2.11            hee7b306_1013    conda-forge
zope-event                4.5.0                    pypi_0    pypi
zope-interface            5.4.0                    pypi_0    pypi
zstd                      1.5.0                h861e0a7_0    conda-forge

opened by htrivino20 6

bizarre behavior of mse metric

>>> r16 = ants.image_read(ants.get_data("r16"))
>>> diff=( r16 - r16 )**2
>>> diff.max()
0.0

>>> r16 = ants.image_read(ants.get_data("r16"))
>>> antspynet.mse( r16, r16 )
9740.324

can you reproduce this? same thing on osx and linux ....

opened by stnava 6

Tensorflow version?

Hi, when trying to run BrainSegmentation with a fresh install of ANTsPyNet i get this error:

RuntimeError: It looks like you are trying to use a version of multi-backend Keras that does not support TensorFlow 2.0. We recommend usingtf.keras, or alternatively, downgrading to TensorFlow 1.14.

When setting tensorflow==1.14 in setup.py I run into a different error with BrainSegmentation:

Loading weights file Traceback (most recent call last): File "Scripts/doBrainTissueSegmentation.py", line 60, in <module> unet_model.load_weights(weights_file_name) File "/Users/vsaase/anaconda3/lib/python3.7/site-packages/Keras-2.2.5-py3.7.egg/keras/engine/saving.py", line 458, in load_wrapper return load_function(*args, **kwargs) File "/Users/vsaase/anaconda3/lib/python3.7/site-packages/Keras-2.2.5-py3.7.egg/keras/engine/network.py", line 1217, in load_weights f, self.layers, reshape=reshape) File "/Users/vsaase/anaconda3/lib/python3.7/site-packages/Keras-2.2.5-py3.7.egg/keras/engine/saving.py", line 1186, in load_weights_from_hdf5_group reshape=reshape) File "/Users/vsaase/anaconda3/lib/python3.7/site-packages/Keras-2.2.5-py3.7.egg/keras/engine/saving.py", line 942, in preprocess_weights_for_loading weights[0] = np.transpose(weights[0], (3, 2, 0, 1)) File "/Users/vsaase/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 639, in transpose return _wrapfunc(a, 'transpose', axes) File "/Users/vsaase/anaconda3/lib/python3.7/site-packages/numpy/core/fromnumeric.py", line 56, in _wrapfunc return getattr(obj, method)(*args, **kwds) ValueError: axes don't match array

which Tensorflow, Numpy and Keras versions are you using? I am on macos 14.14 running in a fresh anaconda virtualenv with python 3.7 also installed antspyx-0.1.8-cp37-cp37m-macosx_10_14_x86_64.whl and tensorflow_probability

thanks, Victor

opened by vsaase 6

resnet3d inconsistency with ANTsRNet

2d appears to be ok at least with default settings.

in 3d, ANTsRNet:

createResNetModel3D(list(128, 128, 128, 1))

gives Total params: 159,536,104

whereas

antspynet.create_resnet_model_3d(( 128, 128, 128, 1)).summary()

gives Total params: 174,141,416

my use case was getting a python equivalent to:

rnmdl =  createResNetModel3D( list(NULL,NULL,NULL,4),
  numberOfClassificationLabels = 1,
  layers = 1:3,
  residualBlockSchedule = c(3,4,6,3), squeezeAndExcite = TRUE,
  lowestResolution = 32, cardinality = 1, mode = "regression" )

which had the same issue ( more params in py ).

opened by stnava 5

WIP: dkt reproducibility

is ready to merge - example shown here is reproducible on OS X ie segmentation exactly the same across runs and looks correct.

import os.path
from os import path
threads = "8"
# set number of threads - this should be optimized per compute instance
os.environ["TF_NUM_INTEROP_THREADS"] = threads
os.environ["TF_NUM_INTRAOP_THREADS"] = threads
os.environ["ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS"] = threads

import ants
import antspynet
img = ants.image_read( "ADNI-016_S_0769-20070820-T1w-000.nii.gz" )
dkt = antspynet.desikan_killiany_tourville_labeling( img,
    template_transform_type='antsRegistrationSyNQuickRepro[a]' )
dkt1 = antspynet.desikan_killiany_tourville_labeling( img,
    template_transform_type='antsRegistrationSyNQuickRepro[a]' )

opened by stnava 5

ANTsPyNet's super resolution models
Hi everyone,

Thanks so much for your effort in providing us such a great tool. I have a few questions regarding the models for super-resolution. I saw that you have implemented several architectures, and that you have also uploaded the pre-trained weights for the Deep Back Projection (DBPN) Network.

Are you going to upload also the pre-trained weights for other models? (SRCNN, ResNet SR...)

Are you going to upload also the training scripts for these models?

Could I know the datasets on which the DBPN has been trained?

Thanks for your time and your help.
opened by carlouic 2
again: "TypeError: Expected `trainable` argument to be a boolean, but got: None"

Hello there ANTSpyNet gurus-

I'm trying to get the DL super-resolution to work. I'm experiencing an error that unfortunately we've already seen #50 and #57 , yet reading through these threads I cannot get the solution hacks to work. Can you advise if there's a new work-around or if I'm doing something incorrectly? In terms of build versions, I installed antspynet via pip last Wednesday July 15.

INFO OF MY SYSTEM: Ubuntu 20.04.4 LTS Python 3.8.13 tensorflow 2.9.1 nvcc --version: Cuda compilation tools, release 10.1, V10.1.243 ls -l /home/sam/.keras/ANTsXNet -rw-rw-r-- 1 sam sam 145251944 Jul 13 16:46 dbpn4x.h5 -rw-rw-r-- 1 sam sam 34878800 Jul 13 17:17 mriSuperResolution.h5

MY COMMANDS: import ants import antspynet import tensorflow as tf os_ds=ants.image_read("/home/sam/Desktop/DLdata_antspynet/sub-masked-ds.nii") ants.plot(os_ds) ptn = antspynet.get_pretrained_network("dbpn4x") print(ptn) srtest = antspynet.utilities.apply_super_resolution_model_to_image(os_ds, ptn, verbose=True)

the plot and the print work but then it breaks at the final line:

MY ERROR: Load model. Traceback (most recent call last): File "sr_antspynet.py", line 20, in srtest = antspynet.utilities.apply_super_resolution_model_to_image(os_ds, ptn, verbose=True) File "/home/sam/miniconda3/lib/python3.8/site-packages/antspynet/utilities/super_resolution_utilities.py", line 279, in apply_super_resolution_model_to_image model = load_model(model) File "/home/sam/miniconda3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "/home/sam/miniconda3/lib/python3.8/site-packages/keras/engine/base_layer.py", line 340, in init raise TypeError( TypeError: Expected trainable argument to be a boolean, but got: None

Finally, if it's useful to know, a co-worker on a similar system was also get through those commands and have it break identically. Any clues would be so incredibly appreciated and thanks again!

-Sam

opened by storrisi42 11
Environment variable for antsxnet_cache_directory

The default cache directory of "~/.keras/ANTsXNet" is troublesome for Singularity users because singularity has to run as the user calling singularity, so users either have to mount their own home directory or ensure every call to get_antsxnet_data or get_pretrained_network specifies antsxnet_cache_directory (with an absolute path) at run time.

I've tried using TF_CACHE_DIR and KERAS_HOME, but they don't work with the tensorflow.keras.utils.get_file function.

So I was thinking of having a variable ANTSXNET_CACHE_DIR, so that data / network downloads would go to

antsxnet_cache_directory if defined in the function call

else

os.environ['ANTSXNET_CACHE_DIR'] if defined

else

~/.keras/ANTsXNet (current default).

Happy to make a PR for this, but wanted to check in first @stnava @ntustison (+anyone else with an opinion)

opened by cookpa 3
DeepFlash for Hippocampal Subfields Segmentation: Reference

Hi,

I am making paper comparing segmentation methods, and as DeepFlash for hippocampal subfields segmentation seems new, I want to include it in my comparison. The website you reference doesn't really include any specific citable material. Does it have any DOI or preprint paper?

Thanks, Clément POIRET

opened by clementpoiret 5
mriSuperresolution along a specific axis of a 3d nifti file

Hi, I have successfully used the built in and trained super-resolution module of antspynet, which scales/resamples to higher resolution with double the dimension across all axis of the original image.

I would like to know how I would be able to use the super resolution module of antsypnet to perform super-resolution along a particular axis of a 3d nifti file?. An example script would be highly appreciated.

opened by limacora45 0

Medical image analysis framework merging ANTsPy and deep learning

Related tags

Overview

ANTsPyNet

Architectures

Image voxelwise segmentation/regression

Image classification/regression

Object detection

Image super-resolution

Registration and transforms

Generative adverserial networks

Clustering

Applications

Miscellaneous

Installation

Publications

Acknowledgments

Comments

Owner

Advanced Normalization Tools Ecosystem

Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimation

The Medical Detection Toolkit contains 2D + 3D implementations of prevalent object detectors such as Mask R-CNN, Retina Net, Retina U-Net, as well as a training and inference framework focused on dealing with medical images.

Pytorch Code for "Medical Transformer: Gated Axial-Attention for Medical Image Segmentation"

Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging

Neuron Merging: Compensating for Pruned Neurons (NeurIPS 2020)

A pytorch-based deep learning framework for multi-modal 2D/3D medical image segmentation

Build a medical knowledge graph based on Unified Language Medical System (UMLS)

TorchIO is a Medical image preprocessing and augmentation toolkit for deep learning. Part of the PyTorch Ecosystem.

Multi-atlas segmentation (MAS) is a promising framework for medical image segmentation

Breaking the Dilemma of Medical Image-to-image Translation

Copy Paste positive polyp using poisson image blending for medical image segmentation

Useful materials and tutorials for 110-1 NTU DBME5028 (Application of Deep Learning in Medical Imaging)

Semi Supervised Learning for Medical Image Segmentation, a collection of literature reviews and code implementations.

ISBI 2022: Cross-level Contrastive Learning and Consistency Constraint for Semi-supervised Medical Image.

[MedIA2021]MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning

Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

[CVPR'21] FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space