Hello,

I want to implement your model in a cell segmentation task. Since I have to segment my images into 4 classes (background, healthy cells, cancer cells and leucocytes) I wanted to ask if your model can be easily modified to support multiclass segmenation. I have little experience in deep learning and python so help would be much appreciated!

When I try to train the MultiResUNet3D model with input shape = (128, 128, 128, 1) and batch size = 1, keras raises this exception:

ResourceExhaustedError: OOM when allocating tensor with shape[1,128,128,128,32] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc
	 [[{{node training/Adadelta/gradients/zeros_70}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

	 [[{{node loss/mul}}]]
Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

also, I used the following function to calculate the gpu memory that keras needs:

# function taked from:
# https://stackoverflow.com/questions/43137288/how-to-determine-needed-memory-of-keras-model

def get_model_memory_usage(batch_size, model):
    shapes_mem_count = 0
    for l in model.layers:
        single_layer_mem = 1
        for s in l.output_shape:
            if s is None:
                continue
            single_layer_mem *= s
        shapes_mem_count += single_layer_mem
    trainable_count = np.sum([K.count_params(p) for p in set(model.trainable_weights)])
    non_trainable_count = np.sum([K.count_params(p) for p in set(model.non_trainable_weights)])
    number_size = 4.0
    if K.floatx() == 'float16':
         number_size = 2.0
    if K.floatx() == 'float64':
         number_size = 8.0
    total_memory = number_size*(batch_size*shapes_mem_count + trainable_count + non_trainable_count)
    gbytes = np.round(total_memory / (1024.0 ** 3), 3)
    return gbytes

and the output when I execute the next code:

# where "model3D" is the MultiResUNet3D model.
get_model_memory_usage(1, model3D) # output 21.628 "GB"

# where "model2D" is the MultiResUNet3D model.
get_model_memory_usage(1, model2D) # output 0.256 "GB"

it is normal that the model does not run? How much GPU memory do I need?

my GPU is a gtx 1080ti (11GB)

Hi there,

I'm about to experiment and use a modified version of your architecture in my master's thesis. I saw in your code that the first convolution layers in MultiResUBlock and ResPath don't use any activation function. What was the reason you decided not to use an activation function here ???

shortcut = conv2d_bn(shortcut, filters, 1, 1,activation=None, padding='same')

Cheers, H

Hi! It is a great honor to see your research paper "MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation". Now I want to reproduce the corresponding code of the paper. Can you provide the pytorch version of the code?

Hi! First of all, thank you very much for sharing your project. It's a great job, congratulations!

I would like to train your network with my own dataset. I've .nii files, so, what I should to do is to stack all my dataset in a numpy array? For example, if I've 20 .nii files with shape [20,256,256], I create an array of [400,256,256]. That's it?

Thank you ver much in advance.

All the best :)

With certain splits of training, validation and test data sets, I can always observe a strange behaviour that does not occur with a different distribution of the same overall data set. Namely, the values on the validation data seem to stand still from the beginning during training, on enormously bad numbers, while the values on the training data keep improving. Unfortunately, it is not really possible to reconstruct which combination of images triggers this behaviour, but it probably really depends on a combination and not on individual images. Is such a behaviour known, or even a solution for it?

Estimated 76.0 GB model memory usage
Train shape: (1166, 672, 672, 1); Val shape: (292, 672, 672, 1); Test shape: (59, 672, 672, 1)
  0%|| 2/15000 [04:11<523:59:44, 125.78s/epoch, val_loss=12.7, val_jaccard_round=0.171, loss=0.43, jaccard_round=0.748]

Hello Nabil,

I check your demo code, i want to to implement 5-fold cross validation in it, and i never found any help anywhere.

Please can you share how i can implement 5-fold cross validation on this code?

Thanks

Hi! It is an honor to read your paper "MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation", and now I want to reproduce the code corresponding to the paper. Unfortunately, the ISIC-17 Dataset download link provided in your demo doesn't seem to work,How should I solve this problem? Words fail me when I wish to express my sincere gratitude to you.Looking forward to your early reply!

In your paper, it appears that you tested this implementation using the ISBI 2012 challenge dataset - Electron Microscopy Image. Do you have a sample demo notebook you could share?

Hi, I wanted to ask about the detail regarding the MultiResBlock architecture. In the Figure 3(a) where you concatenating the generated feature maps. How can you concatenate different size of feature maps. Because as shown in the Figure 3(a), those layers are 3x3, 5x5, and 7x7 which each block have different size of kernel. Can you provide more detail explanation on the Figure 3(a). Thankyou.

Hi,

Thank you very much for your work. I'm trying to implement the MultiResUNet as a generator of a GAN. I would like to know if the definition of trans_conv_2d has any purpose since it is not used. Also, why did you decide not to use dropout?

Thanks!

Hello, I have tried using the MultiRes U-Net Architecture on a different dataset. My accuracy and loss works well. However, the Jaccard index and DICE coefficient remains always zero. Could you please let me know why this might happen and any possible suggestions for this?

Hi,

I have run your network based on the notbook in a project of mine. However, I pondered quite a bit over my validation Jaccard scores outperforming the training score by a large margin. I suspect the answer lies in the rounding of yp that you perform in evaluateModel. From what I can tell, this rounding is not done in the function that is used during training. After removing this rounding the scores matched as expected.

Please let me know if I'm missing the point somewhere, or if you agree with the observation.

Thanks for a superb piece of work!

Arild

Hi @nibtehaz , I've been trying your model for my image segmentation projects (with same settings) but with 365 images which split (0.2) into 292 for training and 73 for testing and manage to get a fairly good results of the dice and Jacard score. But I found out that the jacard score is still at around 0.7 for too many epochs (20-30 epochs) and stop increasing even after 100 epochs. And i found out among those 73 testing images , for some predicted image the jacard score did go up to 0.91 for some only for 0.65 which totally average to around 0.7. I know it should be the dataset problems , I'd like if there is any way to improved the overall jacard score with that same dataset ? Should I just keep training ? Or make some changes to the dataset ?

Dear Author, I did not found any dataloader and prepared code for dataset organization. How we train the model. Moreover, How we can test the performance of the model?