Hi, I tried to run the test code on cityscapes data set, and I changed the crop_size in option.py to 768, but I found the mean IoU was about 60%. Do I need to change the base_size? Or do you know how can I achieve the results reported in your paper? I used pretrained resnet50 provided by you. Thanks a lot!

RuntimeError: Error building extension 'enclib_gpu': [1/4] :/usr/local/cuda-9.0/bin/nvcc -DTORCH_EXTENSION_NAME=enclib_gpu -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/TH -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/THC -I:/usr/local/cuda-9.0/include -I/home/ayx/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options '-fPIC' -std=c++11 -c /home/ayx/ShelfNet-citys/encoding/lib/gpu/encoding_kernel.cu -o encoding_kernel.cuda.o FAILED: encoding_kernel.cuda.o :/usr/local/cuda-9.0/bin/nvcc -DTORCH_EXTENSION_NAME=enclib_gpu -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/TH -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/THC -I:/usr/local/cuda-9.0/include -I/home/ayx/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options '-fPIC' -std=c++11 -c /home/ayx/ShelfNet-citys/encoding/lib/gpu/encoding_kernel.cu -o encoding_kernel.cuda.o /bin/sh: 1: :/usr/local/cuda-9.0/bin/nvcc: not found [2/4] :/usr/local/cuda-9.0/bin/nvcc -DTORCH_EXTENSION_NAME=enclib_gpu -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/TH -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/THC -I:/usr/local/cuda-9.0/include -I/home/ayx/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options '-fPIC' -std=c++11 -c /home/ayx/ShelfNet-citys/encoding/lib/gpu/syncbn_kernel.cu -o syncbn_kernel.cuda.o FAILED: syncbn_kernel.cuda.o :/usr/local/cuda-9.0/bin/nvcc -DTORCH_EXTENSION_NAME=enclib_gpu -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/TH -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/THC -I:/usr/local/cuda-9.0/include -I/home/ayx/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options '-fPIC' -std=c++11 -c /home/ayx/ShelfNet-citys/encoding/lib/gpu/syncbn_kernel.cu -o syncbn_kernel.cuda.o /bin/sh: 1: :/usr/local/cuda-9.0/bin/nvcc: not found [3/4] :/usr/local/cuda-9.0/bin/nvcc -DTORCH_EXTENSION_NAME=enclib_gpu -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/TH -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/THC -I:/usr/local/cuda-9.0/include -I/home/ayx/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options '-fPIC' -std=c++11 -c /home/ayx/ShelfNet-citys/encoding/lib/gpu/roi_align_kernel.cu -o roi_align_kernel.cuda.o FAILED: roi_align_kernel.cuda.o :/usr/local/cuda-9.0/bin/nvcc -DTORCH_EXTENSION_NAME=enclib_gpu -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/TH -I/home/ayx/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/lib/include/THC -I:/usr/local/cuda-9.0/include -I/home/ayx/anaconda3/envs/pytorch/include/python3.6m -D_GLIBCXX_USE_CXX11_ABI=0 --compiler-options '-fPIC' -std=c++11 -c /home/ayx/ShelfNet-citys/encoding/lib/gpu/roi_align_kernel.cu -o roi_align_kernel.cuda.o /bin/sh: 1: :/usr/local/cuda-9.0/bin/nvcc: not found ninja: build stopped: subcommand failed.

I trained Shelfnet realtime model on custom dataset,with 4 classes- road,curb,sidewalk,driveable-fallback,with 4.5k images for training and 80k iterations for training.For the other classes I kept the id as 255.I used the following command for training CUDA_VISIBLE_DEVICES=0 python3 -m torch.distributed.launch --nproc_per_node=2 train.py I got the following result

I am wondering why the result is so bad.The val miou is nan Any suggestion is greatly appreciated. Thank you

作者你好，我看了你的代码和论文，目前shelfnet50 single scale和multi scale在Cityscapes的val集上得到了与你论文里面类似的结果，但是我发现这里有一个问题，你所对比的那些方法，他们在Cityscapes数据集上都是以2048x1024去统计性能的，而我观察你代码发现，你设置Cityscapes数据集的base_size为1024，在做测试的时候你以base_size乘以scale作为最长边去对图像进行缩放，在scale小于2的时候，输入到网络中的图像是永远小于2048x1024的，也就意味着跟其他用2048x1024去统计分割性能的方法没有可比性了，不知道你这边是出于什么考量？ 不好意思打扰了。

I am pretty appreciate your work. I try to re-implement your work, but when I refer to your pytorch code, I don't find any shared-weights operation but it does mentioned in your paper.
I notice that the shared-weights block should appear in every stage in shelfnet's decoder side and the in/out channels of conv2d are not the same, so how can I share the weights of residual block?

Dear @juntang-zhuang,

First of all, thank you for this repo. I am trying to use it to train shelfnet on the Mapillary Vistas Dataset (here you can find my fork). I have succeeded training she Real-Time version of Shelfnet, however the results are pretty bad even after 270000 epochs. The reached mIOU is 34,06%, however in this paper they say that they were able to achieve 49.2% on your model. I have already tried to contact the authors, but I got no response.
Therefore I wanted to ask you if you could give me some tips to improve the training to achieve better results.

Thank you in advance.

Best, Micaela

Hi great work @juntang-zhuang . I have trained the model on a custom dataset(say dataset1),and I also have another dataset(say dataset2),which is similar to dataset1,and hence I want to assign the custom dataset trained model weights as the initial weights for training the model on the dataset2.In the code I did not find where exactly we need to load the pretrained weights for training. Any suggestion is appreciated Thank you

How come you didn't compare to Residual Conv-Deconv Grid Network for Semantic Segmentation? They proposed a multi-path network for semantic segmentation a while ago, which you claim as your main contribution.

Hi, Zhuang. I meet a problem in /ShelfNet18_realtime/train.py

 if it % 1000 == 0:
    ## dump the final model
    save_pth = osp.join(respth, 'shelfnet_model_it_%d.pth'%it)
    # net.cpu()
    # state = net.module.state_dict() if hasattr(net, 'module') else net.state_dict()
    # if dist.get_rank() == 0: torch.save(state, save_pth)
    torch.save(net.module.state_dict(),save_pth)

    if it % 1000 == 0 and it > 0:
        evaluate(checkpoint=save_pth)

when this code block runs in the following line evaluate(checkpoint=save_pth), it appears the problem in /ShelfNet18_realtime/evaluate.py

def evaluate(respth='./res', dspth='/home/cjj/datasets/CityScapes/Fine', checkpoint=None):
...
if checkpoint is None:
        save_pth = osp.join(respth, 'model_final.pth')
    else:
        save_pth = checkpoint
    net.load_state_dict(torch.load(save_pth))
    net.cuda()
    net.eval()
...

and the line net.load_state_dict(torch.load(save_pth)) arises a error like that : RuntimeError: storage has wrong size: expected -4885659930368473377 got 589824

Have you meet this problem like that? Thx

Thanks for your work! I have a problem while running the code ShelfNetlw. The environment configuration I use is as follows: torch 1.0.0 python3.6 and when give the command CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 train.py

The error is:TypeError: init() got an unexpected keyword argument 'find_unused_parameters' so, I tried to delete line76 of train.py (find_unused_parameters=True) But I encountered this situation as shown:

There is something that confuses me, the DistributedDataParallel in pytorch1.0.0 does not have this parameter “find_unused_parameters”，but the lack of find_unused_parameters can create trouble.

Can someone tell me how to solve it? Best not to change the version of pytorch.

Hi I find your code is useful but there is question confuse me, when you test your code on pascal voc val set, your use _val_sync_transform function to crop and pad the image, but if we want to test the network performance on pascal voc test set, we should transform the image size to the original image size. can you share what should i do, thanks for your code.

Hello, I am trying to figure out if there is a way to get the shelfnet network also give the confidence of the segmented class during test time.Basically give how accurate the network thinks the segmentation output is. Any suggestion is greatly appreciated. Thank you

Thank you for the great work! I met a problem with No module named 'inplace_abn' when I run python3 evaluate.py. After installing inplace_abn from official repo, it works still not.

Do you have any advices? THX and have a nice day!

Hi, Thanks a lot for sharing the code. I wanted to export an ONNX model. Hence, I replaced all the synchronized batchnorm with pytorch's batch-norm. However, I observed huge drop in accuracy(~20%). When I dig deeper, I realized that inside the batch-norm kernel, you are taking the absolute value of the weights and adding eps to it. This is functionally different from pytorch's batch-norm.

What is the reason behind this slightly different implementation of batch-norm? Does it help in training or something else?