I am trying to use this model for binary segmentation.

When i pass the Mask as Tensor to muti_bce_loss_fusion I get this error:

    547 def muti_bce_loss_fusion(d0, d1, d2, d3, d4, d5, d6, labels_v):
    548     print(d0.shape)
--> 549     loss0 = bce_loss(d0, labels_v)
    550     loss1 = bce_loss(d1, labels_v)
    551     loss2 = bce_loss(d2, labels_v)

~/anaconda3/envs/seg/lib/python3.7/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    556             result = self._slow_forward(*input, **kwargs)
    557         else:
--> 558             result = self.forward(*input, **kwargs)
    559         for hook in self._forward_hooks.values():
    560             hook_result = hook(self, input, result)

~/anaconda3/envs/seg/lib/python3.7/site-packages/torch/nn/modules/loss.py in forward(self, input, target)
    518 
    519     def forward(self, input, target):
--> 520         return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction)
    521 
    522 

~/anaconda3/envs/seg/lib/python3.7/site-packages/torch/nn/functional.py in binary_cross_entropy(input, target, weight, size_average, reduce, reduction)
   2415 
   2416     return torch._C._nn.binary_cross_entropy(
-> 2417         input, target, weight, reduction_enum)
   2418 
   2419 

RuntimeError: expected dtype Half but got dtype Long

What is the format of the output of the model? What is the expected format for the label?

How could I use this model for binary segmentation?

Hi!

Thank you for your great work with u2net, its awesome!

I have converted the original u2netp model to CoreML to use as .mlmodel file in iOS. It works great but when I test some images I see some differences from executing the original model in python:

Original image:

Original model:

iOS model:

I have tested both models with friends pics and sometimes the difference is huge: python model detects all the body but iOS detects only the face.

Do you know what is happening? Thank you!

I am in the works of doing a web site that has this running in AWS lambda. Front is really simple with Vue.js, upload image --> send to backend --> get it back after this transformation . I have it 100% working locally with Flask and most of the problems with AWS Lambda are solved.

I will post here a link to the website once I am ready (Hopefully this week)

Hi @NathanUA,

I have a library that makes use of your model.

@alfonsmartinez opened an issue about the model result, please, take a look at here: https://github.com/danielgatis/rembg/issues/14

Can you figure out how I can achieve this result without the black fringe?

thanks.

First of all thanks for the amazing work on U-2-net. Now i am trying to train the model from scratch on my own dataset of 60k images which is larger than your dataset. I would like to know how i can increase the model capacity to be able to train on such a dataset.

I have considered replacing the standard rebnconv blocks with residuals as suggested in another issue. What other options i could try? I understand that i need to make the architecture deeper, does this mean that i should make RSU-8 or RSU-9 blocks by adding more convolution layers?

Thanks a lot for you awesome performing model! I'm wondering about scaling and random crop, for training you first scale and then crop to 288x288 and thus the tensor has this size (288), what role does then scaling play here and why you talk about 320x320 as input size instead of 288x288?

RescaleT(320), 
RandomCrop(288),

With your latest model update, upscaling supports different ratios, as it looks like for me, or is only squarish input supported or e.g. 640x480 as well?

According to the requirements of the paper, I flip the DUTS-TR data set offline horizontally, and then train. The code was not modified. epoch_num=450，The number of iterations is 600k to 800K，to test the DUTS-TE test set. However, using your evaluation code, the results Mae and MAXF are not as good as those in the paper.

What's my problem？

I am trying to reproduce your results but having some uninspiring signs at the start. I start with your model with all the settings as you stated in the paper except that I use 15% of the train data as validation every epoch and my batch size is 8. The validation loss after 50 epochs or so stops decreasing and there emerges a noticeable gap between train and validation. I trained for more 40 epochs but validation did not fall lower, it is almost twice of that of the train loss.

The model seems to be overfitting to me. A lower batch-size than yours should cause more regularisation so that should not be the issue.

Can you please give me some advice on how to interpret this and if I should keep going? I know i am not using 100% of data like you but 85% should be suboptimial but similar. Can you share your training curves or anything as such?

@xuebinqin Thanks for nice work . I have trained the u2net model on the custom created person data set . It works perfectly on the images which have person inside it. But it also mask, some parts of those images which have only background , person does not exist inside it . How I can tackle the false result of prediction . I have more than 5k person images with variety of background in the training set . Should I need to add simple background(without person inside it) images into train set. Please suggest me some way to remove false results . thanks

首先感谢大佬对u^2net的开源！ 经过一段时间的使用，我发现u^2net小目标的分割效果没有大目标的好，请问大佬有什么改善的思路吗？ 同时想请问下大佬有考虑对MICCAI 2020 Thyroid Nodule Segmentation Challenge的比赛方案开源吗？现在比赛都是无脑堆efficientnet backbone，很高兴能看到有独立思考的U^2net取得好成绩，也想学习下大佬的思路对模型进行魔改！ 谢谢大佬~

for i, data in enumerate(salobj_dataloader), tells us that model is being trained on one image at one point of time, how can we train on test on multiple images for removing background from a large number of images.

We are testing this model on cpu but the latency is bit longer than expected. We are testing images in a batch of 10 with which issue of latency is raised. If we increase the number of images in batch,the process is getting killed.

What confused me most is the training schema, i.e., in a segmentation way or a GAN way. In the original work, https://github.com/yiranran/APDrawingGAN, the model is trained in a GAN way. I wonder if the U-2-Net is trained in a segmentation way, i.e., directly predicts the sketch which is consisted of 0/1 pixels. Thanks!

Bumps pillow from 8.1.1 to 9.3.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hello, I'm doing research on salient object detection. However, I'm new to the field and deep learning in general. I trained u2net but I don't know how to plot the training curve, and also the values for the evaluation metrics (PR curve, F-measure, MAE, Sm, etc. Could you provide the code to plot the training curve and to get the value for the evaluation metrics, please? Thank you

Hi, Tried to convert the pretrained model to TensorRT with either torch/TensorRT and torch2trt from Nvidia. Despite using the same precision (FP32) the model outputs differ significantly on large images.

Was it something that done successfully by anyone yet?