`/usr/local/lib/python3.5/dist-packages/torch/nn/modules/loss.py:217: UserWarning: NLLLoss2d has been deprecated. Please use NLLLoss instead as a drop-in replacement and see https://pytorch.org/docs/master/nn.html#torch.nn.NLLLoss for more details. warnings.warn("NLLLoss2d has been deprecated. " /usr/local/lib/python3.5/dist-packages/torch/nn/_reduction.py:49: UserWarning: size_average and reduce args will be deprecated, please use reduction='mean' instead. warnings.warn(warning.format(ret)) 08-27 20:15:43.742 Using Cross Entropy Loss /usr/local/lib/python3.5/dist-packages/encoding/nn/syncbn.py:149: EncodingDeprecationWarning: encoding.nn.BatchNorm2d is now deprecated in favor of encoding.nn.SyncBatchNorm. .format('BatchNorm2d', SyncBatchNorm.name), EncodingDeprecationWarning) Traceback (most recent call last):

File "train.py", line 380, in main() File "train.py", line 132, in main net = network.get_net(args, criterion) File "/data/code/GSCNN/network/init.py", line 12, in get_net criterion=criterion, trunk=args.trunk) File "/data/code/GSCNN/network/init.py", line 27, in get_model net = net_func(num_classes=num_classes, trunk=trunk, criterion=criterion) File "/data/code/GSCNN/network/gscnn.py", line 233, in init self.gate1 = gsc.GatedSpatialConv2d(32, 32) File "/data/code/GSCNN/my_functionals/GatedSpatialConv.py", line 36, in init False, _pair(0), groups, bias, 'zeros') TypeError: init() takes 11 positional arguments but 12 were given `

I have tried my best to make sure all the necessary libraries are the right versions: absl-py (0.8.0) astor (0.8.0) certifi (2019.6.16) chardet (3.0.4) cycler (0.10.0) decorator (4.4.0) gast (0.2.2) google-pasta (0.1.7) grpcio (1.23.0) h5py (2.9.0) idna (2.8) imageio (2.5.0) joblib (0.13.2) Keras-Applications (1.0.8) Keras-Preprocessing (1.1.0) kiwisolver (1.1.0) Markdown (3.1.1) matplotlib (3.0.3) networkx (2.3) nose (1.3.7) numpy (1.17.1) opencv-python (4.1.0.25) Pillow (6.1.0) pip (9.0.1) protobuf (3.9.1) pyparsing (2.4.2) python-dateutil (2.8.0) PyWavelets (1.0.3) PyYAML (5.1.2) requests (2.22.0) scikit-image (0.15.0) scikit-learn (0.21.3) scipy (1.1.0) setuptools (20.7.0) six (1.12.0) tensorboard (1.14.0) tensorboardX (1.8) tensorflow (1.14.0) tensorflow-estimator (1.14.0) termcolor (1.1.0) torch (1.0.0) torch-encoding (1.0.1) torchvision (0.2.0) tqdm (4.35.0) urllib3 (1.25.3) Werkzeug (0.15.5) wheel (0.29.0) wrapt (1.11.2)

Do you have any suggestions on how to go about fixing this?

Best regards, Shreyas

Hi, I am not sure the below is an error but at the same time, looking at the train.py evaluate function, it is supposed to output "threshold" and F-Score values, so I do not think the code ran up to the end properly. I can see there is an out of memory error message before processing frames but I do not know if the memory issue was a critical error since the code continued running. The frame section is not clearly looking like an error, but at the same time it could be.

Can you tell me if this is the expected output?

thanks

gscnn_train_eval_run_output.txt

Hi, thanks for the repo. However, I am not able to run my training on the cityscapes dataset. I have around 50 images for training and about 10 for validation and testing each. I have reduced the image resolution to 128x128 and still, it gives the CUDA out of memory error. I am running this on Google Colab which has 12 GB of GPU memory. Can you tell me what I should do to be able to run this model? Any changes that have to be tweaked? @shubhaminnani @tovacinni @varunjampani @davidjesusacu @ShreyasSkandanS

In the install instructions you mention PyTorch (<= 1.0.0 ). I have pytorch 1.1, so does it means it is going to fail for sure or that you did not test with version above 1.0? If the former, is there a reason why such a recent paper/code does not run on pytorch version beyond 1.0? Thanks.

After reading your paper and discussion, we have few questions.

1. In the leaderboard, we can see that you use Mapillary data. However, In the paper of table 6. You compared TKCN and AAF-PSP which didn't use Mapillay data. It is obviously unfair. So what is real result without Maypiilary data and only use fine-data on Cityscapes test set ?
2. Also we didn't find your claim in using Mapillary data in your paper why didn't mention it?
3. In the table 3, Why mIoU is so low when use your ResNet101 backbone according other repo. ResNet101 based deeplabv3+ should be 78+. Repo: https://github.com/speedinghzl/pytorch-segmentation-toolbox

Thanks for your share.

Several questions:

1. Does Eq.(6) miss a minus sign?
2. How to obtain the gradient of an image for the input of the shape stream? soble?
3. The regular steam outputs multi-scale feat-maps. How to obtain the attention maps at a resolution of HxW always?

ValueError: Caught ValueError in replica 0 on device 0. Original Traceback (most recent call last): File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/parallel/parallel_apply.py", line 60, in _worker output = module(*input, **kwargs) File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/home/renshasha/GSCNN-master/network/gscnn.py", line 315, in forward x = self.aspp(m7, acts) File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/home/renshasha/GSCNN-master/network/gscnn.py", line 167, in forward img_features = self.img_conv(img_features) File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/modules/container.py", line 92, in forward input = module(input) File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/modules/batchnorm.py", line 81, in forward exponential_average_factor, self.eps) File "/home/renshasha/anaconda3/envs/aaa/lib/python3.7/site-packages/torch/nn/functional.py", line 1652, in batch_norm raise ValueError('Expected more than 1 value per channel when training, got input size {}'.format(size)) ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 256, 1, 1])

I'm just reading the paper and looking at the code to follow along. In doing so I've gotten a bit confused because there seems to be an extra shape feature map in the code. In networks/gscnn.py on line 274 we have

s1 = F.interpolate(self.dsn1(m1), x_size[2:], mode='bilinear', align_corners=True)

but this is never output or used. Following the paper I would expect this to be the input to the self.res1 unit. However, this is not the case. What is actually put through that is

m1f = F.interpolate(m1, x_size[2:], mode='bilinear', align_corners=True)
cs = self.res1(m1f)

Something is off here as we calculate s1 but do nothing with it. My question is, should s1 replace m1f or should it just be deleted?

When I run 'python train.py --evaluate --snapshot checkpoints/best_cityscapes_checkpoint.pth',CUDA out of memory. My GPU is 11GB of memory,How can I solve this problem，thx.

Looks like the loss implementation does not apply argmax to the semantic segmentation logits, right? But in the paper, equation (4) applies argmax before ConvTri and Gradient computation, right? Which one is the one used to produce the experimental results?

Thank you, Hai

Thank for your great work and share your code,but i have same question,i can't finde the code load wide resnet pretraned model from ImageNet.Can anyone help?thank you.

When I run 'python train.py --evaluate --snapshot checkpoints/best_cityscapes_checkpoint.pth', the bug occurs as:

Traceback (most recent call last): File "train.py", line 381, in main() File "train.py", line 128, in main assert_and_infer_cfg(args) File "E:\Program Files\Anaconda3\envs\newName\E20220227\GSCNN\config.py", line 86, in assert_and_infer_cfg import encoding ModuleNotFoundError: No module named 'encoding'

Then I try to install 'encoding' or search the module 'encoding' in google, but I found nothing. Why does this bug occur? And, how can I settle this problem?

Hi, I met a package bug when trying to evaluate the method by 'python train.py --evaluate --snapshot checkpoints/best_cityscapes_checkpoint.pth'

The bug is : Traceback (most recent call last): File "train.py", line 380, in main() File "train.py", line 127, in main assert_and_infer_cfg(args) File "C:\Users\asd\Desktop\GSCNN-master\config.py", line 86, in assert_and_infer_cfg import encoding File "D:\anconda3\envs\GSCNN0415\lib\site-packages\encoding_init_.py", line 13, in from . import nn, functions, parallel, utils, models, datasets, transforms File "D:\anconda3\envs\GSCNN0415\lib\site-packages\encoding\nn_init_.py", line 12, in from .encoding import * File "D:\anconda3\envs\GSCNN0415\lib\site-packages\encoding\nn\encoding.py", line 18, in from ..functions import scaled_l2, aggregate, pairwise_cosine File "D:\anconda3\envs\GSCNN0415\lib\site-packages\encoding\functions_init_.py", line 2, in from .encoding import * File "D:\anconda3\envs\GSCNN0415\lib\site-packages\encoding\functions\encoding.py", line 14, in from .. import lib File "D:\anconda3\envs\GSCNN0415\lib\site-packages\encoding\lib_init_.py", line 15, in ], build_directory=cpu_path, verbose=False) File "D:\anconda3\envs\GSCNN0415\lib\site-packages\torch\utils\cpp_extension.py", line 644, in load is_python_module) File "D:\anconda3\envs\GSCNN0415\lib\site-packages\torch\utils\cpp_extension.py", line 824, in _jit_compile return _import_module_from_library(name, build_directory, is_python_module) File "D:\anconda3\envs\GSCNN0415\lib\site-packages\torch\utils\cpp_extension.py", line 967, in _import_module_from_library file, path, description = imp.find_module(module_name, [path]) File "D:\anconda3\envs\GSCNN0415\lib\imp.py", line 297, in find_module raise ImportError(_ERR_MSG.format(name), name=name) ImportError: No module named 'enclib_cpu'

It seems the unmatched packages in my conda env: pytorch 1.1.0 python 3.6.13 torch-encoding 1.0.1 torchvision 0.3.0

So, is the torch encoding unmatched? Could you tell me how to fix it. Thank you!

hi, guys, i'm recently using GSCNN for traning my own dataset. I modifed some py files related with reading dataset and ran the training successfully. But computer always crashed and rebooted after training for a while. BTW i used two gpu gtx2080Ti for training. Do you guys have same problem and how to fix this?

GSCNN is a novel idea. However the code is based on wide-resnet. I want to use resnet backbone, Where to get gate from resnet? And the first conv is 7*7 stide 2, where to begin for shape stream.