MOOSE (Multi-organ objective segmentation) a data-centric AI solution that generates multilabel organ segmentations to facilitate systemic TB whole-person research

I am running MOOSE in a patient folder with two subfolders for CT and PET under DCIOM format. However, I am getting this error message:

moose_ct_atlas = ie.segment_ct(ct_file[0], out_dir) File "/export/moose/moose-0.1.0/src/inferenceEngine.py", line 78, in segment_ct out_label = fop.get_files(out_dir, pathlib.Path(nifti_img).stem + '*')[0] IndexError: list index out of range

Any suggestion, please?

Thanks,

Is your feature request related to a problem? Please describe. If the environment variables are not loaded, MOOSE fails silently like so:

✔ Converted DICOM images in /home/user/Data/... to NIFTI
- Only CT data found in folder /home/user/Data/..., MOOSE will construct noncerebral tissue atlas (n=37) based on CT 
- Initiating CT segmentation protocols
- CT image to be segmented: /home/user/Data/...._0000\.nii\.gz                            
✔ Segmented abdominal organs from /home/user/Data/..._0000.nii.gz                                     
Traceback (most recent call last):                                                                                                                                                                                 
    File "/usr/local/bin/moose", line 131, in <module>
        ct_atlas = ie.segment_ct(ct_file[0], out_dir)                                                                                                                                                             
    File "/home/user/Code/MOOSE/src/inferenceEngine.py", line 78, in segment_ct                                                                                                                                        
        out_label = fop.get_files(out_dir, pathlib.Path(nifti_img).stem + '*')[0]                
IndexError: list index out of range

Describe the solution you'd like It would be nice to let the user know that the problem is that the nnUNet_raw_data_base, nnUNet_preprocessed, etc. env variables are not set.

Hi,

I am trying to run MOOSE on a bunch of patients with whole-body CTs. For two of the patient, MOOSE fails with the following error

✔ Segmented psoas from /home/user/Data/....IMA_0000.nii.gz                                              
- Conducting automatic error analysis in similarity space for: /home/user/Data/.../labels/MOOSE-Non-cerebral-tissues-CT-....nii.gz
Traceback (most recent call last):
  File "/usr/local/bin/moose", line 139, in <module>                                                                                                                                                        
    ea.similarity_space(ct_atlas, sim_space_dir, segmentation_error_stats)                                                                                                                                         
  File "/home/user/Code/MOOSE/src/errorAnalysis.py", line 147, in similarity_space
    shape_parameters = iop.get_shape_parameters(split_atlas)
  File "/home/user/Code/MOOSE/src/imageOp.py", line 86, in get_shape_parameters
    label_img = SimpleITK.Cast(SimpleITK.ReadImage(label_image), SimpleITK.sitkInt32)
  File "/home/user/miniconda3/envs/moose/lib/python3.9/site-packages/SimpleITK/extra.py", line 346, in ReadImage
    return reader.Execute()
  File "/home/user/miniconda3/envs/moose/lib/python3.9/site-packages/SimpleITK/SimpleITK.py", line 8015, in Execute
    return _SimpleITK.ImageFileReader_Execute(self)
RuntimeError: Exception thrown in SimpleITK ImageFileReader_Execute: /tmp/SimpleITK/Code/IO/src/sitkImageReaderBase.cxx:97:
sitk::ERROR: The file "/home/user/Data/.../labels/sim_space/similarity-space/MOOSE-Split-unified-PET-CT-atlas.nii.gz" does not exist.

Do you know what could be the problem if the file not existing? It works for the other patients.

Need to manually start again:

Calculated SUV image for SUV extraction!

• Brain found in field-of-view of PET/CT data...
• Cropping brain from PET image using the aligned CT brain mask Traceback (most recent call last): File "/usr/local/bin/moose", line 214, in cropped_pet_brain = iop.crop_image_using_mask(image_to_crop=pet_file[0], File "/home/mz/Documents/Softwares/MOOSE-V.1.0/src/imageOp.py", line 228, in crop_image_using_mask bbox = np.asarray(label_shape_filter.GetBoundingBox(1)) File "/usr/local/lib/python3.8/dist-packages/SimpleITK/SimpleITK.py", line 36183, in GetBoundingBox return _SimpleITK.LabelShapeStatisticsImageFilter_GetBoundingBox(self, label) RuntimeError: Exception thrown in SimpleITK LabelShapeStatisticsImageFilter_GetBoundingBox: /tmp/SimpleITK-build/ITK-prefix/include/ITK-5.2/itkLabelMap.hxx:151: ITK ERROR: LabelMap(0x9547bd0): No label object with label 1.

Currently MOOSE is running on server configuration. So there is a good chance that the user is using a DGX or so. In that case, it would make sense to fully utilise the capabilities of the hardware. Similar to falcon, moose should run in parallel based on the hardware capabilities.

The following error occurred after using Moose with dynamic datasets of Vision lung cancer patients. All other segmentations and SUV extraction properly worked. No error occurred after re-running Moose with the corresponding static dataset.

Brain found in field-of-view of PET/CT data...                         
- Cropping brain from PET image using the aligned CT brain mask
Traceback (most recent call last):
  File "/usr/local/bin/moose", line 215, in <module>
    cropped_pet_brain = iop.crop_image_using_mask(image_to_crop=pet_file[0],
  File "/home/mz/Documents/Softwares/MOOSE/src/imageOp.py", line 237, in crop_image_using_mask
    out_of_bounds = upper_bounds >= img_dim
ValueError: operands could not be broadcast together with shapes (3,) (4,)

MOOSE fails with index error when trying to run on WSL, due to wrong installation. There is no moose-files folder created when the algorithm is installed.

Steps to reproduce the behavior: Install through WSL as described in github.

Moose-files folder should be created when installed, and moose should run as required.

Screenshots of the errors:

Windows 11 22H2

When MOOSE fails to infer the dataset, the command is stopped and the folders are left with temporary files given in this structure:

Newly created folders: CT, PT, labels, stats, temp and 2 .JSON files.

In order to clean these datasets and make them executable again, it would be nice to have a command to revert them into their original states. The command which can manually be used is listed here.

root@meta-zoo:/home/mz/Documents/Projects/"Work-Directory"/faulty# find -maxdepth 2 -name CT -exec rm -rf {} \; root@meta-zoo:/home/mz/Documents/Projects/"Work-Directory"/faulty# find -maxdepth 2 -name PT -exec rm -rf {} \; root@meta-zoo:/home/mz/Documents/Projects/"Work-Directory"/faulty# find -maxdepth 2 -name labels -exec rm -rf {} \; root@meta-zoo:/home/mz/Documents/Projects/"Work-Directory"/faulty# find -maxdepth 2 -name temp -exec rm -rf {} \; root@meta-zoo:/home/mz/Documents/Projects/"Work-Directory"/faulty# find -maxdepth 2 -name stats -exec rm -rf {} \;

Right now MOOSE breaks when there is no brain in the PET image. The elegant way would be to figure out if there is a brain in the FOV of PET and initiate the segmentation protocols accordingly. It seems to be quite hard to determine if a given image has a brain in the field of view with hand-engineered features. The smartest way would be to generate a MIP or the middle slice of the PET image (if given) and use a 2D CNN based binary classifier for figuring if the brain is in the FOV or not.

The game plan is the following:

• [x] Extract the middle slice (coronal plane)

• [x] Convert it from DICOM to .png and transform the PET intensities between 0-255 (Graylevels)

• [x] Curate 80 slices (50 PET with no brain, 50 PET with a brain) and perform the training.

• [x] Implement a 2D CNN binary-classifier (PyTorch <3 fastai)

• [x] Make sure the data augmentations of the 2D CNN have random cropping

• [x] Then use the trained model to infer whether a given volume has a brain or not.

Problem. Since MOOSE is pretty much used in servers, it might be worthwhile to have a Docker Image for MOOSEv0.1.0.

Solution Need to make one with the docker image hosted at IBM cloud.

MOOSE fails when presented with a dynamic PET in the latest version. It works as expected with static 3D images.

MOOSE probably doesn't need to do anything special with the 4D dynamic images, but it should probably still produce the segmented CT output. Additionally, it would be great to have a registration between the CT and the final frame of the PET. Motion correction of the PET could then be performed with FALCON, and mapped back to the CT.

Hello,

would it be possible to skip a patient and process the next one in case of an error (e.g. empty CT dir) and not stop the process?

And then maybe in the end you get a list of the patient IDs that failed.

Dear MOOSE team,

I mentioned the following issue in another issue and wanted to create a new one for it:

I don't know if adding the env variables to `.bashrc` is the best place to do it. Some users might use zsh and others might use nnUnet seperately.  

Originally posted by @chris-clem in https://github.com/QIMP-Team/MOOSE/issues/42#issuecomment-1286930959.

As a quick solution, I added a env_vars.sh file in the MOOSE repo dir that I source instead of .bashrc. In the meantime, I have searched how people are handling the problem in general and found the following possibilities:

1. Create a .env file in the repo dir and load it with python-dotenv as explained here.
2. Create a .env file in the repo dir and recommend users to use direnv, which then automatically loads the env variables when changing in the MOOSE dir.
3. Recommend users to create a MOOSE conda environment and enable loading and unloading the env vars when activating/ deactivating the conda environment as described here.

The downside of 1. is that it requires a new dependency, the downside of 2. that it requires a new program, and the downside of 3. is that it requires conda for managing the environment.

What do you think is the best option?

Problem

Inference is a tad bit slow when it comes to large datasets.

Solution Performance gains can be achieved by using Intel's Neural Compressor: https://github.com/intel/neural-compressor/tree/master/examples/pytorch/image_recognition/3d-unet/quantization/ptq/eager. And Intel has already provided an example on how to do so. So we just need to implement this for getting a lean model (still need to check the performance gains)

*Alternate solution

Is to bring in a fast resampling function (torch or others...)

Problem MOOSE is based on nnUNet and the current inference takes a lot of memory on total-body datasets (uEXPLORER/QUADRA, upper limit: 256 GB). This is not a normal memory usage for most of the users. The memory usage bottleneck is explained here: https://github.com/MIC-DKFZ/nnUNet/issues/896

Solution The solution seems to be to find a 'faster/memory efficient' resampling scheme than the skimage resampling scheme. People have already suggested solutions for speed, based on https://pytorch.org/docs/stable/generated/torch.nn.functional.interpolate.html and an elaborate description can be found here: https://github.com/MIC-DKFZ/nnUNet/issues/1093.

But the memory consumption is still a problem. @dhaberl @Keyn34 : Consider these alternative options of Nvidia's cuCIM cucim.skimage.transform.resize in combination with Dask for block processing (chunks consume way less memory and I have used this for kinetic modelling).

Impact This would result in a faster inference time and hopefully also obviates memory bottleneck for MOOSE and for any model inference via nnUNet.

Analysis request for prostate cancer cohort as follows:

• [x] MOOSE cohort -> Validation of Segmentations by me
  • [ ] Extract PET-Parameters from MOOSEd Segments
• [x ] Delete all hand drawn PET-Segmentations starting with cubic*
• [ ] Merge all the remaining Segmentations (pb*, sv*, pln*...) on a patient level by the following convention:
  • [ ] all Segmentations to a Master_VOI -> extract PET-Parameters (SUVmax, mean... + Metabolic Tumor volume)
  • [ ] VOIs named: pb* + sv* -> Prostate_Sum_VOI -> extract PET-Parameters (SUVmax, mean... + Metabolic Tumor volume)
  • [ ] VOIs named: dln* + pln* + rln* -> Lymph_node_Sum_VOI -> extract PET-Parameters (SUVmax, mean... + Metabolic Tumor volume)
  • [ ] VOIs named: bone* -> Bone_Sum_VOI -> extract PET-Parameters (SUVmax, mean... + Metabolic Tumor volume)
  • [ ] VOIs named: adrenal* + liver* + pleura* + lung* + rectum* + skin* + peritoneal* + org* + organ* + psoas* + testis* + lung* + cavern* -> Organ_Sum_VOI -> extract PET-Parameters (SUVmax, mean... + Metabolic Tumor volume)

In case of mucosal congestion in the nasal cavity and paranasal sinuses -> missclassification as skeletal muscles. This appears often but I think the effects are minor, hence MINOR bug. All instances recorded