Implementation for paper: Self-Regulation for Semantic Segmentation

Dong ZHANG

Last update: Nov 21, 2022

Related tags

Deep Learning SR-SS

Overview

Self-Regulation for Semantic Segmentation

This is the PyTorch implementation for paper Self-Regulation for Semantic Segmentation, ICCV 2021.

Citing SR-SS

@article{zhang2021sr,
  title={Self-Regulation for Semantic Segmentation},
  author={Dong, Zhang and Hanwang, Zhang and Jinhui, Tang and Xiansheng, Hua and Qianru, Sun},
  journal={International Conference on Computer Vision (ICCV)},
  year={2021}
}

Contact

If you have any questions, don't hesitate to contact: [email protected].

Installation Instructions

Clone this repo:

git clone [email protected]:dongzhang89/SR-SS.git
cd SR-SS

Model Train:

python train.py

Model Test:

python test.py

Comments

Bug in ClassLayer?

In the forward method of class ClassLayer in lib/models.deeplab.py:

def forward(self, x):
        x = self.layer(x)
        x = self.classification(x)
        pred = x.view(x.shape[0], -1)
        pred = torch.sigmoid(self.fc(pred))
        return x, pred

the returned x does not represent feature maps anymore because self.classification is an AdaptiveAvgPooling2D with output_size=1, and thus returns a tensor with shape BxCx1x1. Then this is resized to the size of the image later, which doesn't seem to make sense.

Shouldn't it be like this?

def forward(self, x):
        x = self.layer(x)
        pred = self.classification(x)
        pred = pred.view(pred.shape[0], -1)
        pred = torch.sigmoid(self.fc(pred))
        return x, pred

opened by fjean 1

When I try to run your code, there are many bugs

I just run follow your readme and modify my data path. I meet: torch.Size([1, 21, 315, 315]) torch.Size([1, 21, 315, 315]) torch.Size([1, 21, 315, 315]) torch.Size([1, 21, 315, 315]) torch.Size([1, 21, 315, 315]) label: torch.Size([1, 315, 315]) tensor([[0.6681, 0.5503, 0.4859, 0.3670, 0.4030, 0.4132, 0.3604, 0.5969, 0.5407, 0.4653, 0.5853, 0.6330, 0.4966, 0.3281, 0.4031, 0.5070, 0.6057, 0.4516, 0.4128, 0.5618, 0.3800]], device='cuda:0', grad_fn=<SigmoidBackward>) torch.Size([1, 21, 315, 315]) torch.Size([1, 21, 315, 315]) torch.Size([1, 21, 315, 315]) torch.Size([1, 21, 315, 315]) label: torch.Size([1, 315, 315]) [W TensorIterator.cpp:918] Warning: Mixed memory format inputs detected while calling the operator. The operator will output contiguous tensor even if some of the inputs are in channels_last format. (function operator()) Time:2021-09-29 22:47:34 [iter 0/40000] loss: 354.8209, lr: 1.000000e-02 tensor([[nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan]], device='cuda:0', grad_fn=<SigmoidBackward>) torch.Size([1, 21, 270, 270]) torch.Size([1, 21, 270, 270]) torch.Size([1, 21, 270, 270]) torch.Size([1, 21, 270, 270]) torch.Size([1, 21, 270, 270]) label: torch.Size([1, 270, 270]) THCudaCheck FAIL file=/pytorch/aten/src/THC/generic/THCTensorMath.cu line=19 error=710 : device-side assert triggered Traceback (most recent call last): File "trainval.py", line 166, in <module> loss2_segmentation = get_seg_loss(out, label) File "/home/hadoop-vacv/cephfs/data/lijinlong/codes/SEMISSS/20210922/SR-SS/lib/models/losses.py", line 132, in get_seg_loss loss += criterion(m(preds[i]), label) File "/home/hadoop-vacv/cephfs/data/lijinlong/local/anaconda3/envs/mmdet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/hadoop-vacv/cephfs/data/lijinlong/local/anaconda3/envs/mmdet/lib/python3.8/site-packages/torch/nn/modules/loss.py", line 211, in forward return F.nll_loss(input, target, weight=self.weight, ignore_index=self.ignore_index, reduction=self.reduction) File "/home/hadoop-vacv/cephfs/data/lijinlong/local/anaconda3/envs/mmdet/lib/python3.8/site-packages/torch/nn/functional.py", line 2220, in nll_loss ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index) RuntimeError: cuda runtime error (710) : device-side assert triggered at /pytorch/aten/src/THC/generic/THCTensorMath.cu:19 /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [0,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [1,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [2,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [3,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [4,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [5,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [6,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [7,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [8,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [9,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [10,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [11,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [12,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [13,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [14,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [15,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [16,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [17,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [18,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [19,0,0] Assertioninput_val >= zero && input_val <= onefailed. /pytorch/aten/src/ATen/native/cuda/Loss.cu:106: operator(): block: [0,0,0], thread: [20,0,0] Assertioninput_val >= zero && input_val <= onefailed.

opened by TyroneLi 0
About the trained models

Many thanks for your excellent work. I am interested in it. I notice that you have not released the trained models. Could you please provide one of the trained models for our visualization?

opened by ShihuaHuang95 1

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

ADE20k Semantic segmentation with MAE Getting started Install the mmsegmentation

97 Dec 17, 2022

Implementation of SETR model, Original paper: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.

SETR - Pytorch Since the original paper (Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.) has no official

112 Dec 16, 2022

PixelPick This is an official implementation of the paper "All you need are a few pixels: semantic segmentation with PixelPick."

PixelPick This is an official implementation of the paper "All you need are a few pixels: semantic segmentation with PixelPick." [Project page] [Paper

59 Sep 25, 2022

Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation Created by Zeyu HU Introduction This work is based on our paper VMNet: Voxel-Mes

82 Dec 27, 2022

Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Pixel-Level Cycle Association This is the Pytorch implementation of our NeurIPS 2020 Oral paper Pixel-Level Cycle Association: A New Perspective for D

87 Oct 19, 2022

Official and maintained implementation of the paper "OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data" [BMVC 2021].

OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data Christoph Reich, Tim Prangemeier, Özdemir Cetin & Heinz Koeppl | Pr

23 Sep 21, 2022

Code for the ICCV 2021 Workshop paper: A Unified Efficient Pyramid Transformer for Semantic Segmentation.

Unified-EPT Code for the ICCV 2021 Workshop paper: A Unified Efficient Pyramid Transformer for Semantic Segmentation. Installation Linux, CUDA=10.0,

29 Aug 23, 2022

Public repository of the 3DV 2021 paper "Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds"

Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds Björn Michele1), Alexandre Boulch1), Gilles Puy1), Maxime Bucher1) and Rena

15 Dec 22, 2022

This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection

Bridge-damage-segmentation This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge c

5 Dec 7, 2022

Implementation for paper: Self-Regulation for Semantic Segmentation

Related tags

Overview

Self-Regulation for Semantic Segmentation

Citing SR-SS

Contact

Installation Instructions

You might also like...

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Implementation of SETR model, Original paper: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.

PixelPick This is an official implementation of the paper "All you need are a few pixels: semantic segmentation with PixelPick."

Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Official and maintained implementation of the paper "OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data" [BMVC 2021].

Code for the ICCV 2021 Workshop paper: A Unified Efficient Pyramid Transformer for Semantic Segmentation.

Public repository of the 3DV 2021 paper "Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds"

This is the code repository for the paper A hierarchical semantic segmentation framework for computer-vision-based bridge column damage detection

Comments

Bug in ClassLayer?

When I try to run your code, there are many bugs

About the trained models

Owner

Dong ZHANG

Code for our paper Domain Adaptive Semantic Segmentation with Self-Supervised Depth Estimation

Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

Pytorch implementation of SenFormer: Efficient Self-Ensemble Framework for Semantic Segmentation

Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Self-supervised Augmentation Consistency for Adapting Semantic Segmentation (CVPR 2021)

ST++: Make Self-training Work Better for Semi-supervised Semantic Segmentation

Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021)

PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

TorchDistiller - a collection of the open source pytorch code for knowledge distillation, especially for the perception tasks, including semantic segmentation, depth estimation, object detection and instance segmentation.