Hi,

I have a datase with 2 classes in VOC format. as I realized, you have prepared the fine tuning by some flags. Correct me if I'm wrong.

Besides, my GPU is GTX1060 with 6G memory. Does the calculated memory consumption belongs to the full 21 classes original VOC? I mean can I train the model on this small dataset?

Nice work porting the model. I found that your evaluation code is wrong. You are evaluating on an image by image basis and summing up IoUs across the val. set. That is not how mean IoU is computed. It is accumulated over pixels. Refer to the FCN code here to see what I mean: https://github.com/shelhamer/fcn.berkeleyvision.org/blob/master/score.py

If I change your evaluation to the correct eval. script, your trained model gets a mIoU of only 72.1%. Also, the deeplab resnet-101 model you had ported into torch only gives 75.4% as opposed to the original 76.3% from CAFFE. This might be due to the different preprocessing you do compared to the deeplab guys or might be due to small errors in porting the model.

It will be great if you can confirm this and put out a log in the README saying you are fixing your eval. script.

Thanks for the amazing work. I am trying to use your model description for another segmentation problem.

But when I run

python train.py

this is the error log that I get.

Traceback (most recent call last): File "train.py", line 219, in images, label = get_data_from_chunk_v2(chunk) File "train.py", line 113, in get_data_from_chunk_v2 labels = [resize_label_batch(gt,i) for i in [a,a,b,a]] File "train.py", line 113, in labels = [resize_label_batch(gt,i) for i in [a,a,b,a]] File "train.py", line 65, in resize_label_batch label_resized = np.zeros((size,size,1,label.shape[3])) TypeError: 'float' object cannot be interpreted as an integer

Additional Info: I am running python 3.5

Excuse me , when i run 'python train.py',a mistake happened as follow:

File "train.py", line 222, in model.load_state_dict(saved_state_dict) File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 331, in load_state_dict .format(name)) KeyError: 'unexpected key "Scale.conv1.weight" in state_dict'

I use the coco pretrained model 'MS_DeepLab_resnet_pretrained_COCO_init.pth' to fine tune by voc , hope for response , Thank you !

RuntimeError: sizes do not match at /b/wheel/pytorch-src/torch/lib/THC/THCTensorCopy.cu:31

I have finetuning this model to train on my custom dataset of images. The groundtruth has only two labels [0 and 255]. However when I test my image using the eval2.py script, I get the following error:

{'--gpu0': '0', '--help': False, '--snapPrefix': 'VOC12_scenes_', '--testGTpath': '/mnt/VTSRAID01/SAMI_EXPERIMENTS/Segmentation/DataForAnalytics/GIS_ALL_IMAGES/BinaryResizedGroundtruthPng/', '--testIMpath': '/mnt/VTSRAID01/SAMI_EXPERIMENTS/Segmentation/DataForAnalytics/GIS_ALL_IMAGES/ResizedOriginalImages/', '--visualize': True} VOC12_scenes_ Traceback (most recent call last): File "evalpyt2.py", line 87, in model.load_state_dict(saved_state_dict) File "/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.py", line 335, in load_state_dict own_state[name].copy_(param) RuntimeError: sizes do not match at /b/wheel/pytorch-src/torch/lib/THC/THCTensorCopy.cu:31

I have crosschecked and the sizes of my input image and the groundtruth do match. I am not sure what is causing this error.

Any help would be much appreciated.

Hi, thanks for the work implementing the model and training script.

I'm attempting to train on optical flow RGB image data with binary segmentation masks (where 0=background and 1=foreground). However, no matter my choice of hyperparameters or number of iterations (20k/40k/80k), the loss generally steadily decreases but the resulting model predicts all background pixels for input images at test time.

I've confirmed that the pretrained model segments correctly, so there's something wrong in the training process. The weights are non-zero but argmax always seems to choose class 0. Do you have any idea what might be wrong?

I'm using a GeForce GTX 1080 and Python 2.7 and am not encountering any memory or other such errors during training.

Why do you use 20000 iteration size with only batch_size=1? In this way, each training image is only passed for 2 times to get the final model.

Another question is that I have run evalpyt2.py and your trained VOC12_scenes_16000.pth model to test. But I only got a validation performance of 66.7%. Do you know why it is so low? Thanks!

Hi, Thank you @isht7 for writing this code. I am having problems in the memory used by the code. If I use a batch size of 1, the memory consumed is around 7-8 GB. I have only 1 GPU and hence I cannot increase the batch size further. However, when I used this library - https://github.com/speedinghzl/Pytorch-Deeplab which implements Deeplabv2 Resnet 101, the batch size can be increased to 10. Isn't this unusual? Could you tell me of any changes I need to make to your code so that I can increase batch size? My GPU has 11.1 GB of memory. Thanking you in anticipation. Regards, Omkar.

Hi, great code!

Could you specify exactly which dataset corresponds to train_aug.txt and val.txt and where to get it ? It is the augmented VOC2012, right?

Hi, the parameters of BatchNorm layer in resnet101 is fixed by here

But the running_mean and running_var is also need to be fix, so I think we need to set BatchNorm to eval mode, not just fix parameters (weight and bias)

Hello I am using anaconda environment to execute your train code. But I am getting below error: /home/abhash/anaconda3/lib/python3.6/site-packages/torch/nn/modules/upsampling.py:180: UserWarning: nn.UpsamplingBilinear2d is deprecated. Use nn.Upsample instead. warnings.warn("nn.UpsamplingBilinear2d is deprecated. Use nn.Upsample instead.") Traceback (most recent call last): File "train.py", line 222, in out = model(images) File "/home/abhash/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 325, in call result = self.forward(*input, **kwargs) File "/home/abhash/Documents/Abhash/MG/deeplab-resnet/pytorch-deeplab-resnet/deeplab_resnet.py", line 198, in forward out.append(self.interp3(self.Scale(x2))) # for 0.75x scale File "/home/abhash/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 325, in call result = self.forward(*input, **kwargs) File "/home/abhash/anaconda3/lib/python3.6/site-packages/torch/nn/modules/upsampling.py", line 181, in forward return super(UpsamplingBilinear2d, self).forward(input) File "/home/abhash/anaconda3/lib/python3.6/site-packages/torch/nn/modules/upsampling.py", line 79, in forward return F.upsample(input, self.size, self.scale_factor, self.mode) File "/home/abhash/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 1375, in upsample return _functions.thnn.UpsamplingBilinear2d.apply(input, _pair(size), scale_factor) File "/home/abhash/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/thnn/upsampling.py", line 277, in forward ctx.output_size[1], TypeError: CudaSpatialUpSamplingBilinear_updateOutput received an invalid combination of arguments - got (int, torch.cuda.FloatTensor, torch.cuda.FloatTensor, float, float), but expected (int state, torch.cuda.FloatTensor input, torch.cuda.FloatTensor output, int outputHeight, int outputWidth)

Also, I am new to pytorch, so let pardon me if I am missing any obvious thing here. P.S.: anaconda3, python3.6 with GPU CUDA: nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2016 NVIDIA Corporation Built on Tue_Jan_10_13:22:03_CST_2017 Cuda compilation tools, release 8.0, V8.0.61

Why is the result on the val set more than three points different from that published by the author? If I want to use it in my WSSS job，will it have a big impact？ Thanks！！

hello, the code you use to read the ground truth of dataset is

    gt_temp = cv2.imread(os.path.join(gt_path,piece+'.png'))[:,:,0]
    gt_temp[gt_temp == 255] = 0

why there is "[:,:,0]" for the ground truth? thank you very much! and when I use cv2 to read the ground truth of pascal voc dataset,

import cv2
import numpy as np
img_cv= cv2.imread('/Users/zhangying/Desktop/2007_000129.png')
img_array_cv = img_cv[:,:,0]
print('use the method CV\n ',np.unique(img_array_cv))

the output is

use the method CV
  [  0 128 192]

however the sum of the categories is 21 for voc dataset(128>20,192>20, is it reasonable?). How do you think about this problem? Thanks a lot!

I just go through the model in MS_Deeplab. In forward pass, the input image x has been pass through Resnet 4 times with different scales. But according to original paper, multi scale should happen after layer5 in resnet. May I know why implementation this way?

Hi to all! I have never used docopt package before. To be able to execute the train.py file, which argument should I input to docstr variable? Im getting an error at "args = docopt(docstr, version='v0.1')"

The error I get is as belows:

  File "/media/sinem/LENOVO/deeplab-resnet-pytorch/train.py", line 41, in <module>
    args = docopt(docstr, version='v0.1')
  File "/usr/local/lib/python2.7/dist-packages/docopt.py", line 558, in docopt
    DocoptExit.usage = printable_usage(doc)
  File "/usr/local/lib/python2.7/dist-packages/docopt.py", line 468, in printable_usage
    raise DocoptLanguageError('"usage:" (case-insensitive) not found.')
docopt.DocoptLanguageError: "usage:" (case-insensitive) not found. 

Do you have a suggestion on how i can solve this?

I installed the latest version: docopt-0.6.2.tar.gz

Cheers, sinem.

I have a question about the performance. "This is in comparision to 75.54% that is acheived by using train_iter_20000.caffemodel released by authors, which can be replicated by running this file . The .pth model converted from .caffemodel using the first section also gives 75.54% mean IOU. " But when I checked the model file and the trained .pth file, I found that the model applied ASPP instead of LargeFOV, which means the model performance should have achieved 76.35% as depicted in the paper. Why the performance is 75.54?