Cancer metastasis detection with neural conditional random field (NCRF)

Overview

Baidu Logo

NCRF

This repository contains the code and data to reproduce the main results from the paper:

Yi Li and Wei Ping. Cancer Metastasis Detection With Neural Conditional Random Field. Medical Imaging with Deep Learning (MIDL), 2018.

If you find the code/data is useful, please cite the above paper:

@inproceedings{li2018cancer,
    title={Cancer Metastasis Detection With Neural Conditional Random Field},
    booktitle={Medical Imaging with Deep Learning},
    author={Li, Yi and Ping, Wei},
    year={2018}
}

If you have any quesions, please post it on github issues or email at [email protected]

Prerequisites

  • Python (3.6)

  • Numpy (1.14.3)

  • Scipy (1.0.1)

  • PyTorch (0.3.1)/CUDA 8.0 The specific binary wheel file is cu80/torch-0.3.1-cp36-cp36m-linux_x86_64.whl. I havn't tested on other versions, especially 0.4+, wouldn't recommend using other versions.

  • torchvision (0.2.0)

  • PIL (5.1.0)

  • scikit-image (0.13.1)

  • OpenSlide 3.4.1(Please don't use 3.4.0 as some potential issues found on this version)/openslide-python (1.1.0)

  • matplotlib (2.2.2)

  • tensorboardX Standard along tensorboard that also works for PyTorch. This is mostly used in monitoring the training curves.

  • QuPath Although not directly relevant to training/testing models, I found it very useful to visualize the whole slide images.

Most of the dependencies can be installed through pip install with version number, e.g.

pip install 'numpy==1.14.3'

Or just simply

pip install numpy

A requirements.txt file is also provided, so that you can install most of the dependencies at once:

pip install -r requirements.txt -i https://pypi.python.org/simple/

For PyTorch please consider downloading the specific wheel binary and use

pip install torch-0.3.1-cp36-cp36m-linux_x86_64.whl

Data

Whole slide images

The main data are the whole slide images (WSI) in *.tif format from the Camelyon16 challenge. You need to apply on Camelyon16 for data access, and once it's approved, you can download from either Google Drive, or Baidu Pan. Note that, one slide is usually ~100Kx100K pixels at level 0 and 1GB+ on disk. There are 400 slides in total, together about 700GB+. So make sure you have enough disk space. The tumor slides for training are named as Tumor_XXX.tif, where XXX ranges from 001 to 110. The normal slides for training are named as Normal_XXX.tif, where XXX ranges from 001 to 160. The slides for testing are named as Test_XXX.tif where XXX ranges from 001 to 130.

Once you download all the slides, please put all the tumor slides and normal slides for training under one same directory, e.g. named /WSI_TRAIN/.

Update

It seems the whole slide image *tif files are now application free to download at GigaDB. But still please contact the Camelyon16 organizers for data usage.

Annotations

The Camelyon16 organizers also provides annotations of tumor regions for each tumor slide in xml format. I've converted them into some what simpler json format, located under NCRF/jsons. Each annotation is a list of polygons, where each polygon is represented by its vertices. Particularly, positive polygons mean tumor region and negative polygons mean normal regions. You can also use the following command to convert the xml format into the json format

python NCRF/wsi/bin/camelyon16xml2json.py Tumor_001.xml Tumor_001.json

Patch images

Although the original 400 WSI files contain all the necessary information, they are not directly applicable to train a deep CNN. Therefore, we have to sample much smaller image patches, e.g. 256x256, that a typical deep CNN can handle. Efficiently sampling informative and representative patches is one of the most critical parts to achieve good tumor detection performance. To ease this process, I have included the coordinates of pre-sampled patches used in the paper for training within this repo. They are located at NCRF/coords. Each one is a csv file, where each line within the file is in the format like Tumor_024,25417,127565 that the last two numbers are (x, y) coordinates of the center of each patch at level 0. tumor_train.txt and normal_train.txt contains 200,000 coordinates respectively, and tumor_valid.txt and normal_valid.txt contains 20,000 coordinates respectively. Note that, coordinates of hard negative patches, typically around tissue boundary regions, are also included within normal_train.txt and normal_valid.txt. With the original WSI and pre-sampled coordinates, we can now generate image patches for training deep CNN models. Run the four commands below to generate the corresponding patches:

python NCRF/wsi/bin/patch_gen.py /WSI_TRAIN/ NCRF/coords/tumor_train.txt /PATCHES_TUMOR_TRAIN/
python NCRF/wsi/bin/patch_gen.py /WSI_TRAIN/ NCRF/coords/normal_train.txt /PATCHES_NORMAL_TRAIN/
python NCRF/wsi/bin/patch_gen.py /WSI_TRAIN/ NCRF/coords/tumor_valid.txt /PATCHES_TUMOR_VALID/
python NCRF/wsi/bin/patch_gen.py /WSI_TRAIN/ NCRF/coords/normal_valid.txt /PATCHES_NORMAL_VALID/

where /WSI_TRAIN/ is the path to the directory where you put all the WSI files for training as mentioned above, and /PATCHES_TUMOR_TRAIN/ is the path to the directory to store generated tumor patches for training. Same naming applies to /PATCHES_NORMAL_TRAIN/, /PATCHES_TUMOR_VALID/ and /PATCHES_NORMAL_VALID/. By default, each command is going to generate patches of size 768x768 at level 0 using 5 processes, where the center of each patch corresponds to the coordinates. Each 768x768 patch is going to be further split into a 3x3 grid of 256x256 patches, when the training algorithm that leverages CRF comes into play.

Note that, generating 200,000 768x768 patches using 5 processes took me about 4.5 hours, and is about 202GB on disk.

Model

NCRF The core idea of NCRF is taking a grid of patches as input, e.g. 3x3, using CNN module to extract patch embeddings, and using CRF module to model their spatial correlations. The CNN module is adopted from the standard ResNet released by torchvision (https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py). The major difference is during the forward pass that 1. the input tensor has one more dimension, 2. use the CRF module to smooth the logit of each patch using their embeddings.

def forward(self, x):
    """
    Args:
        x: 5D tensor with shape of
        [batch_size, grid_size, 3, crop_size, crop_size],
        where grid_size is the number of patches within a grid (e.g. 9 for
        a 3x3 grid); crop_size is 224 by default for ResNet input;
    Returns:
        logits, 2D tensor with shape of [batch_size, grid_size], the logit
        of each patch within the grid being tumor
    """
    batch_size, grid_size, _, crop_size = x.shape[0:4]
    # flatten grid_size dimension and combine it into batch dimension
    x = x.view(-1, 3, crop_size, crop_size)

    x = self.conv1(x)
    x = self.bn1(x)
    x = self.relu(x)
    x = self.maxpool(x)

    x = self.layer1(x)
    x = self.layer2(x)
    x = self.layer3(x)
    x = self.layer4(x)

    x = self.avgpool(x)
    # feats means features, i.e. patch embeddings from ResNet
    feats = x.view(x.size(0), -1)
    logits = self.fc(feats)

    # restore grid_size dimension for CRF
    feats = feats.view((batch_size, grid_size, -1))
    logits = logits.view((batch_size, grid_size, -1))

    if self.crf:
        logits = self.crf(feats, logits)

    logits = torch.squeeze(logits)

    return logits

The CRF module only has one trainable parameter W for pairwise potential between patches. You can plot the W from the ckpt file (see next section) of a trained CRF model by

python NCRF/wsi/bin/plot_W.py /PATH_TO_MODEL/best.ckpt

When the CRF model is well trained, W typically reflects the relative spatial positions between different patches within the input grid. For more details about the model, please refer to our paper.

Training

With the generated patch images, we can now train the model by the following command

python NCRF/wsi/bin/train.py /CFG_PATH/cfg.json /SAVE_PATH/

where /CFG_PATH/ is the path to the config file in json format, and /SAVE_PATH/ is where you want to save your model in checkpoint(ckpt) format. Two config files are provided at NCRF/configs, one is for ResNet-18 with CRF

{
 "model": "resnet18",
 "use_crf": true,
 "batch_size": 10,
 "image_size": 768,
 "patch_size": 256,
 "crop_size": 224,
 "lr": 0.001,
 "momentum": 0.9,
 "data_path_tumor_train": "/PATCHES_TUMOR_TRAIN/",
 "data_path_normal_train": "/PATCHES_NORMAL_TRAIN/",
 "data_path_tumor_valid": "/PATCHES_TUMOR_VALID/",
 "data_path_normal_valid": "/PATCHES_NORMAL_VALID/",
 "json_path_train": "NCRF/jsons/train",
 "json_path_valid": "NCRF/jsons/valid",
 "epoch": 20,
 "log_every": 100
}

Please modify /PATCHES_TUMOR_TRAIN/, /PATCHES_NORMAL_TRAIN/, /PATCHES_TUMOR_VALID/, /PATCHES_NORMAL_VALID/ respectively to your own path of generated patch images. Please also modify NCRF/jsons/train and NCRF/jsons/valid with respect to the full path to the NCRF repo on your machine. The other config file is for ResNet-18 without CRF (the baseline model).

By default, train.py use 1 GPU (GPU_0) to train model, 2 processes for load tumor patch images, and 2 processes to load normal patch images. On one GTX 1080Ti, it took about 5 hours to train 1 epoch, and 4 days to finish 20 epoches. You can also use tensorboard to monitor the training process

tensorboard --logdir /SAVE_PATH/

training_acc Typically, you will observe the CRF model consistently achieves higher training accuracy than the baseline model.

train.py will generate a train.ckpt, which is the most recently saved model, and a best.ckpt, which is the model with the best validation accuracy. We also provide the best.ckpt of pretained resnet18_base and resnet18_crf at NCRF/ckpt.

Testing

Tissue mask

The main testing results from a trained model for WSI analysis is the probability map that represents where on the WSI the model thinks is tumor region. Naively, we can use a sliding window fashion that predicts the probability of all the patches being tumor or not across the whole slide image. But since most part of the WSI is actually white background region, lots of computation is wasted in this sliding window fashion. Instead, we first compute a binary tissue mask that represent each patch is tissue or background, and then tumor prediction is only performed on tissue region. A typical WSI and its tissue mask looks like this (Test_026) tissue_mask To obtain the tissue mask of a given input WSI, e.g. Test_026.tif, run the following command

python NCRF/wsi/bin/tissue_mask.py /WSI_PATH/Test_026.tif /MASK_PATH/Test_026.npy

where /WSI_PATH/ is the path to the WSI you are interested, and /MASK_PATH/ is the path where you want to save the generated tissue mask in numpy format. By default, the tissue mask is generated at level 6, corresponding to the inference stride of 64, i.e. making a prediction every 64 pixels at level 0.

The tissue mask of Test_026_tissue_mask.npy at level 6 is attached for comparison. Note that, when you plot it using matplotlib.pyplot.imshow, please transpose it.

Probability map

With the generated tissue mask, we can now obtain the probability map of a given WSI, e.g. Test_026.tif, using a trained model:

python NCRF/wsi/bin/probs_map.py /WSI_PATH/Test_026.tif /CKPT_PATH/best.ckpt /CFG_PATH/cfg.json /MASK_PATH/Test_026.npy /PROBS_MAP_PATH/Test_026.npy

where /WSI_PATH/ is the path to the WSI you are interested. /CKPT_PATH/ is where you saved your trained model and best.ckpt corresponds to the model with the best validation accuracy. /CFG_PATH/ is the path to the config file of the trained model in json format, and is typically the same as /CKPT_PATH/. /MASK_PATH/ is where you saved the generated tissue mask. /PROBS_MAP_PATH/ is where you want to save the generated probability map in numpy format.

By defautl, probs_map.py use GPU_0 for interence, 5 processes for data loading. Note that, although we load a grid of patches, e.g. 3x3, only the predicted probability of the center patch is retained for easy implementation. And because of this heavy computational overhead, it takes 0.5-1 hour to obtain the probability map of one WSI. We are thinking about developing more efficient inference algorithm for obtaining probability maps. probability_map This figure shows the probability maps of Test_026 with different settings: (a) original WSI, (b) ground truth annotation, (c) baseline method, (d) baseline method with hard negative mining, (e) NCRF with hard negative mining. We can see the probability map from the baseline method typically has lots of isolated false positives. Hard negative mining significantly reduces the number of false positives for the baseline method, but the probability density among the ground truth tumor regions is also decreased, which decreases model sensitivity. NCRF with hard negative mining not only achieves low false positives but also maintains high probability density among the ground truth tumor regions with sharp boundaries.

The probability map of Test_026_probs_map.npy at level 6 is attached for comparison. Note that, when you plot it using matplotlib.pyplot.imshow, please transpose it.

Tumor localization

We use non-maximal suppression (nms) algorithm to obtain the coordinates of each detectd tumor region at level 0 given a probability map.

python NCRF/wsi/bin/nms.py /PROBS_MAP_PATH/Test_026.npy /COORD_PATH/Test_026.csv

where /PROBS_MAP_PATH/ is where you saved the generated probability map, and /COORD_PATH/ is where you want to save the generated coordinates of each tumor regions at level 0 in csv format. There is an optional command --level with default value 6, and make sure it's consistent with the level used for the corresponding tissue mask and probability map.

FROC evaluation

With the coordinates of tumor regions for each test WSI, we can finally evaluate the average FROC score of tumor localization.

python NCRF/wsi/bin/Evaluation_FROC.py /TEST_MASK/ /COORD_PATH/

/TEST_MASK/ is where you put the ground truth tif mask files of the test set, and /COORD_PATH/ is where you saved the generated tumor coordinates. Evaluation_FROC.py is based on the evaluation code provided by the Camelyon16 organizers with minor modification. Note, Test_049 and Test_114 are excluded from the evaluation as noted by the Camelyon16 organizers.

Comments
  • slide/mask mismatch

    slide/mask mismatch

    If I follow the instructions for testing as given in readme. I get the following error

    Traceback (most recent call last):
      File "wsi/bin/probs_map.py", line 163, in <module>
        main()
      File "wsi/bin/probs_map.py", line 159, in main
        run(args)
      File "wsi/bin/probs_map.py", line 115, in run
        args, cfg, flip='NONE', rotate='NONE')
      File "wsi/bin/probs_map.py", line 87, in make_dataloader
        flip=flip, rotate=rotate),
      File "/media/udion/a2c5c487-f939-4b82-a348-86b3d1bdb024/udion_home/Projects/NCRF/wsi/bin/../../wsi/data/wsi_producer.py", line 42, in __init__
        self._preprocess()
      File "/media/udion/a2c5c487-f939-4b82-a348-86b3d1bdb024/udion_home/Projects/NCRF/wsi/bin/../../wsi/data/wsi_producer.py", line 55, in _preprocess
        .format(X_slide, X_mask, Y_slide, Y_mask))
    Exception: Slide/Mask dimension does not match , X_slide / X_mask : 98304 / 1536, Y_slide / Y_mask : 103936 / 2048
    

    what's the issue?

    opened by udion 30
  • Some issues about the reproducing results

    Some issues about the reproducing results

    When I tried to reproduce your codes, there are less-than-perfect results. For example, below is the raw test_001 tiff tif_raw_convert_img And after the whole training with the ResNet18-CRF, I got the test prob map result as: probmap_convert_img while the ground-truth mask is something like:(since the camelyon16 organizers didn't provide the test GT with the format of tiff anymore, I transferred the raw tiff test file and xml file to the tiff mask with the ASAP software manually.) npy_mask_convert_img

    And I have just followed your test steps and evaluated the average FROC score for the whole test set, and got this: froc_npy

    However, the result is not at all satisfying.

    And is there any other trick in your preprocess, postprocess, or the training process?

    Here is the prob map of test_026:

    probmap_convert_img_026

    opened by yux94 21
  • probability map semantics

    probability map semantics

    the output of the command for generating probability map, like

    python NCRF/wsi/bin/probs_map.py /WSI_PATH/Test_012.tif /CKPT_PATH/best.ckpt /CFG_PATH/cfg.json /MASK_PATH/Test_012.npy /PROBS_MAP_PATH/Test_012.npy
    

    generates numpy file for the probability map, but the semantics of numpy file are bit odd, how do I intereprete the values?

    for example, in the above case, If I print the max and min values of test_012.npy, then I get 0.05732796713709831 and 0.0 Which seems odd to me, as I was expecting max probability to be 1 (or close to one), but such low value represents that at every pixel probability of detecting cancer is so small?

    opened by udion 19
  • openslide.lowlevel.OpenSlideError: TIFFRGBAImageGet failed

    openslide.lowlevel.OpenSlideError: TIFFRGBAImageGet failed

    JPEGLib: Not a JPEG file: starts with 0x00 0x00. Traceback (most recent call last): File "wsi/bin/patch_gen.py", line 48, in (args.patch_size, args.patch_size)).convert('RGB') File "/home/list-2018/.conda/envs/tensorflow/lib/python3.5/site-packages/openslide/init.py", line 223, in read_region level, size[0], size[1]) File "/home/list-2018/.conda/envs/tensorflow/lib/python3.5/site-packages/openslide/lowlevel.py", line 259, in read_region _read_region(slide, buf, x, y, level, w, h) File "/home/list-2018/.conda/envs/tensorflow/lib/python3.5/site-packages/openslide/lowlevel.py", line 196, in _check_error raise OpenSlideError(err) openslide.lowlevel.OpenSlideError: TIFFRGBAImageGet failed

    opened by Hukongtao 17
  • device-side assert triggered

    device-side assert triggered

    I got RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/aten/src/THC/generic/THCTensorCopy.c:70 when calculate the loss(loss = loss_fn(output, target)), something wrong with my label?

    opened by LXYTSOS 12
  • 利用CAMELYON16提供的Annotations在測試時無法正確計算FROC

    利用CAMELYON16提供的Annotations在測試時無法正確計算FROC

    每個xml所產生的mask.tif經過computeEvaluationMask皆只有返回一個腫瘤 ex . test_004.xml具有3個腫瘤註釋,但是生成的mask tif文件返回1個腫瘤 xml to tif reader = mir.MultiResolutionImageReader() mr_image = reader.open('../images/test_021.tif') annotation_list = mir.AnnotationList() xml_repository = mir.XmlRepository(annotation_list) xml_repository.setSource('test_021.xml') xml_repository.load() annotation_mask = mir.AnnotationToMask() camelyon17_type_mask = False label_map = {'metastases': 1, 'normal': 2} if camelyon17_type_mask else {'_0': 1, '_1': 1, '_2': 0} conversion_order = ['metastases', 'normal'] if camelyon17_type_mask else ['_0', '_1', '_2'] output_path = "test_021_mask.tif" annotation_mask.convert(annotation_list, output_path, mr_image.getDimensions(), mr_image.getSpacing(), label_map, conversion_order)

    evaluation mask def computeEvaluationMask(maskDIR, resolution, level): """Computes the evaluation mask.

    Args:
        maskDIR:    the directory of the ground truth mask
        resolution: Pixel resolution of the image at level 0
        level:      The level at which the evaluation mask is made
        
    Returns:
        evaluation_mask
    """
    slide = openslide.open_slide(maskDIR)
    dims = slide.level_dimensions[level]
    pixelarray = np.zeros(dims[0]*dims[1], dtype='uint')
    pixelarray = np.array(slide.read_region((0,0), level, dims))
    distance = nd.distance_transform_edt(255 - pixelarray[:,:,0])
    Threshold = 75/(resolution * pow(2, level) * 2) # 75µm is the equivalent size of 5 tumor cells
    binary = distance < Threshold
    filled_image = nd.morphology.binary_fill_holes(binary)
    evaluation_mask = measure.label(filled_image, connectivity = 2) 
    return evaluation_mask
    
    opened by christinesu12 11
  • About how to generater Normal_*.json

    About how to generater Normal_*.json

    Hi,

    I was wondering how to generater Normal_*. json files?

    And if it was generatered by the normal tissue mask? can you share the process detail. Thank you!

    Kind regards,

    opened by wilmerwang 8
  • CRF implementation in Keras is not  not giving  good results

    CRF implementation in Keras is not not giving good results

    Hi, I am a student and working on Camelyon’16 as my Master’s project. I was going through your very impressive paper - Yi Li and Wei Ping. Cancer Metastasis Detection With Neural Conditional Random Field. Medical Imaging with Deep Learning (MIDL), 2018. And, found that you have implemented CRF in your code on top of Resnet 18. So far I am using Resnet50 but my FROC score is not going up from 0.55.

    So, I have decided to use your approach and re-implemented your code in Keras (backend-Tensorflow). But the performance of the trained model is not even close to your results. Best FROC of Resnet18+CRF trained model-0.55. Lot of FPs are coming. My Resnet18 is taken from https://github.com/raghakot/keras-resnet

    My queries-

    • My training loss- BCE- started from 1.16 and finally settled to 0.8639 and validation loss 0.8528. It is right or loss should go further down. I have run for 30+ epochs, but the loss remains the same(plateau). Don’t know why? (please refer attached image below)

    • Weight plot doesn’t look closer to what you have shown in your paper. In your, case all the positional patch weights W[0,0,0,0] <0,W[0,1,0,1] <0..W[2,2,2,2] <0 but I am not seeing that. As per equation, these will not affect the final predictions. (please refer attached image below)

    Can you help me in understanding why loss is not going further down? It gets plateau after a certain number of epochs and after that no effect even with Cyclic Learning rate [ 1e-4,1e-5,1e-7]. This behavior is common across many models – rennet50/101/18 + Inception V3. Please help me to solve these problems. I shall be thankful to you.

    Training Configration config

    TensorBoard ACC/LOSS plot image002

    image006

    Just Validation loss and training loss ( after 16 epochs ) -- Orange is val-loss, Blue is Training BCE loss val_train_loss

    Weight plots- plot across epochs-

    weight_epochs

    Just one of the epoch(16) weight map from which heatmap is generated image009

    Heatmaps Test_001.tiff ( cam'16 test data set) Results from my trained model- at level 8

    001

    Results for your model-at level 6 test_001_crfbaidu

    Clearly, your model performs far better than my trained model.

    I have matched my CRF implementation in kears with yours in pyTorch. For the same input in both the model, I am getting the same output.

    Please help me to reproduce your results in Kears+ TF.

    opened by OpenCv30 7
  • Cannot get correct FROC with resnet18 baseline ckpt

    Cannot get correct FROC with resnet18 baseline ckpt

    I tried to calculate FROC using your resnet18_base.ckpt, and I got: Avg FP = 0.25 Sensitivity = 0.6061946902654868 Avg FP = 0.5 Sensitivity = 0.6769911504424779 Avg FP = 1 Sensitivity = 0.7345132743362832 Avg FP = 2 Sensitivity = 0.7876106194690266 Avg FP = 4 Sensitivity = 0.8185840707964602 Avg FP = 8 Sensitivity = 0.8628318584070797 Avg Sensivity = 0.7477876106194691 I have excluded the test_114.tif in the test set, but there is a gap between my results and the paper said (0.7825). But I got the correct FROC using resnet18_crf.ckpt. If the resnet18 baseline ckpt given in the project is same as which you used to calculate the FROC in paper? Thanks a lot.

    opened by xray-pku 6
  • Codes for coordinate generation

    Codes for coordinate generation

    Could you release the codes for coordinates generation? Now the patch size is fixed to be 768x768. Once it changes, we need to generate new coordinates. Also need to extract and save a new set of extracted images, which is spatially expensive.

    opened by SamitHuang 6
  • mask of Test_026 inference stride

    mask of Test_026 inference stride

    In Tissue mask part, you said you generate the mask of Test_026.tif in level 6 corresponding to the inference stride of 64, but I found the level 6 downsample factor is 57.375, dose it means the inference stride is nearly 57 instead of 64?

    opened by YingdiZhang 5
  • Bump pillow from 8.1.1 to 9.3.0

    Bump pillow from 8.1.1 to 9.3.0

    Bumps pillow from 8.1.1 to 9.3.0.

    Release notes

    Sourced from pillow's releases.

    9.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    9.3.0 (2022-10-29)

    • Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [wiredfool]

    • Initialize libtiff buffer when saving #6699 [radarhere]

    • Inline fname2char to fix memory leak #6329 [nulano]

    • Fix memory leaks related to text features #6330 [nulano]

    • Use double quotes for version check on old CPython on Windows #6695 [hugovk]

    • Remove backup implementation of Round for Windows platforms #6693 [cgohlke]

    • Fixed set_variation_by_name offset #6445 [radarhere]

    • Fix malloc in _imagingft.c:font_setvaraxes #6690 [cgohlke]

    • Release Python GIL when converting images using matrix operations #6418 [hmaarrfk]

    • Added ExifTags enums #6630 [radarhere]

    • Do not modify previous frame when calculating delta in PNG #6683 [radarhere]

    • Added support for reading BMP images with RLE4 compression #6674 [npjg, radarhere]

    • Decode JPEG compressed BLP1 data in original mode #6678 [radarhere]

    • Added GPS TIFF tag info #6661 [radarhere]

    • Added conversion between RGB/RGBA/RGBX and LAB #6647 [radarhere]

    • Do not attempt normalization if mode is already normal #6644 [radarhere]

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Project dependencies may have API risk issues

    Project dependencies may have API risk issues

    Hi, In NCRF, inappropriate dependency versioning constraints can cause risks.

    Below are the dependencies and version constraints that the project is using

    numpy==1.14.3
    scipy==1.0.1
    torchvision==0.2.0
    pillow==8.1.1
    scikit-image==0.13.1
    openslide-python==1.1.0
    matplotlib==2.2.2
    tensorboardX
    

    The version constraint == will introduce the risk of dependency conflicts because the scope of dependencies is too strict. The version constraint No Upper Bound and * will introduce the risk of the missing API Error because the latest version of the dependencies may remove some APIs.

    After further analysis, in this project, The version constraint of dependency scipy can be changed to >=0.12.0,<=1.7.3. The version constraint of dependency pillow can be changed to ==9.2.0. The version constraint of dependency pillow can be changed to >=2.0.0,<=9.1.1. The version constraint of dependency scikit-image can be changed to >=0.13.0,<=0.18.3.

    The above modification suggestions can reduce the dependency conflicts as much as possible, and introduce the latest version as much as possible without calling Error in the projects.

    The invocation of the current project includes all the following methods.

    The calling methods from the scipy
    scipy.ndimage.morphology.binary_fill_holes
    scipy.ndimage.distance_transform_edt
    
    The calling methods from the pillow
    PIL.Image.open
    
    The calling methods from the scikit-image
    skimage.filters.threshold_otsu
    skimage.measure.points_in_poly
    
    The calling methods from the all methods
    m.bias.data.zero_
    x.get
    computeITCList
    pid.self._annotations.inside_polygons
    torch.bmm
    self.layer4
    set
    torch.load
    scipy.ndimage.distance_transform_edt
    skimage.measure.points_in_poly
    m.weight.data.normal_
    time.strftime
    FP_probs.append
    args.patch_size.args.patch_size.args.level.y.x.slide.read_region.convert.save
    self.layer3
    Probs.append
    numpy.float32.img.np.array.transpose.transpose
    self._resolution.np.log2.is_integer
    torch.norm
    numpy.random.seed
    i.positive_vertices.astype
    self.bn2
    numpy.transpose
    numpy.where
    open
    compute_FP_TP_Probs
    os.mkdir
    numpy.zeros
    numpy.float32.img.np.array.transpose
    self.conv1
    computeEvaluationMask
    torch.nn.Sequential
    logging.basicConfig
    numpy.amax
    line.strip.split
    Thresh.unlisted_FPs.np.asarray.sum
    matplotlib.pyplot.plot
    args.level.slide.level_dimensions.args.level.slide.read_region.convert
    matplotlib.pyplot.show
    numpy.rot90
    skimage.measure.label
    Xcorr.append
    numpy.round
    copy.deepcopy
    range
    self.bn3
    matplotlib.pyplot.xlabel
    os.path.join
    torch.nn.Linear
    numpy.random.randint
    numpy.asarray.append
    wsi.data.annotation.Formatter.camelyon16xml2json
    int.strip
    numpy.interp
    torch.nn.Parameter
    json_dict.append
    logits.sigmoid
    skimage.filters.gaussian.max
    opts_list.append
    ckpt.cpu.numpy
    self._preprocess
    xml.etree.ElementTree.parse
    self.fc
    self.Bottleneck.super.__init__
    numpy.random.rand
    str
    numpy.asarray
    numpy.logical_not
    self.maxpool
    skimage.measure.regionprops
    matplotlib.pyplot.figure
    int.Y.X.np.round.astype.transpose
    torch.nn.ReLU
    self.avgpool
    sys.path.append
    math.sqrt
    each.endswith
    wsi.data.annotation.Annotation.from_json
    self.ResNet.super.__init__
    argparse.ArgumentParser
    print
    openslide.open_slide.read_region
    open.write
    inxml.ET.parse.getroot.findall
    matplotlib.pyplot.imshow
    multiprocessing.Pool.map
    self._image_size.self._image_size.y.x.self._slide.read_region.convert
    run
    os.path.dirname
    feats.view.view
    Polygon.inside
    skimage.color.rgb2hsv
    float
    sorted
    self.downsample
    format
    torch.nn.MaxPool2d
    list
    int.Y.X.np.round.astype.transpose.tolist
    torch.zeros
    numpy.max
    argparse.ArgumentParser.parse_args
    self.BasicBlock.super.__init__
    skimage.filters.gaussian
    ckpt.cpu.numpy.reshape
    torch.nn.Conv2d
    i.negative_vertices.astype
    openslide.OpenSlide.read_region
    shutil.copyfile
    csvDIR.open.readlines
    multiprocessing.Pool
    int.i.negative_vertices.astype.tolist
    self._polygons_negative.append
    os.path.abspath
    numpy.fliplr
    plotFROC
    logging.info
    inxml.ET.parse.getroot
    Y.X.np.round.astype
    matplotlib.pyplot.subplot
    self._color_jitter
    numpy.log2
    self.layer1
    self.crf
    self.layer2
    torch.nn.BatchNorm2d
    Isolated_Tumor_Cells.append
    torch.transpose
    numpy.mean
    sum
    computeFROC
    line.rstrip
    self.CRF.super.__init__
    args.patch_size.args.patch_size.args.level.y.x.slide.read_region.convert
    sys.stdout.flush
    len
    self._make_layer
    map
    torchvision.transforms.ColorJitter
    wsi.model.layers.CRF
    numpy.load
    argparse.ArgumentParser.add_argument
    matplotlib.pyplot.figure.suptitle
    Ycorr.append
    line.strip
    conv3x3
    block
    layers.append
    self._slide.read_region
    pow
    Thresh.unlisted_TPs.np.asarray.sum
    super
    torch.sum
    matplotlib.pyplot.ylabel
    logits.clone
    int
    self.avgpool.size
    readCSVContent
    enumerate
    os.path.exists
    x.vertices
    multiprocessing.Value
    self._coords.append
    self.bn1
    PIL.Image.open
    skimage.filters.threshold_otsu
    line.rstrip.split
    m.weight.data.fill_
    numpy.save
    self.avgpool.view
    self.relu
    total_TPs.append
    json.load
    self._polygons_positive.append
    numpy.float32.img.np.array.transpose.rotate
    ResNet
    os.listdir
    main
    json.dump
    numpy.array
    annotation.findall
    self.conv2
    self.modules
    torch.squeeze
    scipy.ndimage.morphology.binary_fill_holes
    self.conv3
    torch.nn.AvgPool2d
    openslide.OpenSlide
    multiprocessing.Lock
    torch.squeeze.view
    torch.load.cpu
    open.close
    wsi.data.annotation.Annotation
    Exception
    Polygon
    int.i.positive_vertices.astype.tolist
    openslide.open_slide
    isinstance
    

    @developer Could please help me check this issue? May I pull a request to fix it? Thank you very much.

    opened by PyDeps 0
  • Can you explain how the code calculates the FROC score?

    Can you explain how the code calculates the FROC score?

    I want to plot and calculate FROC for 3 WSIs (Test_001, Test_065, and Test_079 from Camelyon16), but the result is nan. I can't understand how do you calculate FROC. We have a CSV file that shows coordinates and probabilities of tumors generated by the model and ground truth mask. Is this ground truth mask have a particular property? What are the values of the ground truth mask? 0 or 1? What is ITC_labels in Evaluation_FROC.py? How do you generate a ground truth mask of WSI from annotations? Is there a particular way or the same as the way provided by ASAP?

    opened by alibalapour 0
  • Have an error with data.cuda(async=True)

    Have an error with data.cuda(async=True)

    When I ran the script to get a probability map of WSI, this error occurred: File "NCRF/wsi/bin/probs_map.py", line 54 data = Variable(data.cuda(async=True), volatile=True) ^ SyntaxError: invalid syntax

    opened by alibalapour 0
Releases(v1.0)
Owner
Baidu Research
Baidu Research
Baidu Research
This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

The Neural Process Family This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CN

DeepMind 892 Dec 28, 2022
Patient-Survival - Using Python, I developed a Machine Learning model using classification techniques such as Random Forest and SVM classifiers to predict a patient's survival status that have undergone breast cancer surgery.

Patient-Survival - Using Python, I developed a Machine Learning model using classification techniques such as Random Forest and SVM classifiers to predict a patient's survival status that have undergone breast cancer surgery.

Nafis Ahmed 1 Dec 28, 2021
Editing a Conditional Radiance Field

Editing Conditional Radiance Fields Project | Paper | Video | Demo Editing Conditional Radiance Fields Steven Liu, Xiuming Zhang, Zhoutong Zhang, Rich

Steven Liu 216 Dec 30, 2022
Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening

Deep Neural Networks Improve Radiologists' Performance in Breast Cancer Screening Introduction This is an implementation of the model used for breast

null 757 Dec 30, 2022
Random-Afg - Afghanistan Random Old Idz Cloner Tools

AFGHANISTAN RANDOM OLD IDZ CLONER TOOLS Install $ apt update $ apt upgrade $ apt

MAHADI HASAN AFRIDI 5 Jan 26, 2022
Receptive Field Block Net for Accurate and Fast Object Detection, ECCV 2018

Receptive Field Block Net for Accurate and Fast Object Detection By Songtao Liu, Di Huang, Yunhong Wang Updatas (2021/07/23): YOLOX is here!, stronger

Liu Songtao 1.4k Dec 21, 2022
CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution

CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution This is the official implementation code of the paper "CondLaneNe

Alibaba Cloud 311 Dec 30, 2022
Official PyTorch code for WACV 2022 paper "CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows"

CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows WACV 2022 preprint:https://arxiv.org/abs/2107.1

Denis 156 Dec 28, 2022
Instance-conditional Knowledge Distillation for Object Detection

Instance-conditional Knowledge Distillation for Object Detection This is a MegEngine implementation of the paper "Instance-conditional Knowledge Disti

MEGVII Research 47 Nov 17, 2022
Cancer Drug Response Prediction via a Hybrid Graph Convolutional Network

DeepCDR Cancer Drug Response Prediction via a Hybrid Graph Convolutional Network This work has been accepted to ECCB2020 and was also published in the

Qiao Liu 50 Dec 18, 2022
Predict Breast Cancer Wisconsin (Diagnostic) using Naive Bayes

Naive-Bayes Predict Breast Cancer Wisconsin (Diagnostic) using Naive Bayes Downloading Data Set Use our Breast Cancer Wisconsin Data Set Also you can

Faeze Habibi 0 Apr 6, 2022
To Design and Implement Logistic Regression to Classify Between Benign and Malignant Cancer Types

To Design and Implement Logistic Regression to Classify Between Benign and Malignant Cancer Types, from a Database Taken From Dr. Wolberg reports his Clinic Cases.

Astitva Veer Garg 1 Jul 31, 2022
SCI-AIDE : High-fidelity Few-shot Histopathology Image Synthesis for Rare Cancer Diagnosis

SCI-AIDE : High-fidelity Few-shot Histopathology Image Synthesis for Rare Cancer Diagnosis Pretrained Models In this work, we created synthetic tissue

Emirhan Kurtuluş 1 Feb 7, 2022
PN-Net a neural field-based framework for depth estimation from single-view RGB images.

PN-Net We present a neural field-based framework for depth estimation from single-view RGB images. Rather than representing a 2D depth map as a single

null 1 Oct 2, 2021
This is the official implementation code repository of Underwater Light Field Retention : Neural Rendering for Underwater Imaging (Accepted by CVPR Workshop2022 NTIRE)

Underwater Light Field Retention : Neural Rendering for Underwater Imaging (UWNR) (Accepted by CVPR Workshop2022 NTIRE) Authors: Tian Ye†, Sixiang Che

jmucsx 17 Dec 14, 2022
PyTorch implementation for OCT-GAN Neural ODE-based Conditional Tabular GANs (WWW 2021)

OCT-GAN: Neural ODE-based Conditional Tabular GANs (OCT-GAN) Code for reproducing the experiments in the paper: Jayoung Kim*, Jinsung Jeon*, Jaehoon L

BigDyL 7 Dec 27, 2022
Random Walk Graph Neural Networks

Random Walk Graph Neural Networks This repository is the official implementation of Random Walk Graph Neural Networks. Requirements Code is written in

Giannis Nikolentzos 38 Jan 2, 2023
An implementation for Neural Architecture Search with Random Labels (CVPR 2021 poster) on Pytorch.

Neural Architecture Search with Random Labels(RLNAS) Introduction This project provides an implementation for Neural Architecture Search with Random L

null 18 Nov 8, 2022
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Jan 3, 2023