Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentation

Related tags

Deep Learning URN
Overview

Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentation

Introduction

This is a PyTorch implementation of Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentation (AAAI2022), based on mmsegmentation. Please refer the classification phase to PMM and refer the segmentation phase to WSSS_MMSeg.

In this papper, we mitigate the noise of pseudo-mask in segmentation phase via uncertainty from response scaling which simulates the behavior of noise. This technique is applicable to all weakly-supervised semantic segmentation methods based on fully-supervised semantic segmentation.

Uncertainty visualization uncertainty visualization

Framework visualization framework visualization

Preparation

(Extract code of BaiduYun: mtci)

Datasets and pretrained weights

VOC12 OneDrive, BaiduYun; COCO14 BaiduYun; Pretrained weights OneDrive, BaiduYun

Pseduo-masks from classification phase

Pseudo-masks (if you want to skip cls phase), VOC12 OneDrive, COCO14 BaiduYun

Intermediate segmentation weights for uncertainty and cyclic pseudo-mask

Intermediate weights (if you want to skip first segmentation), BaiduYun

Released segmentation weights for test and visualization

Released weights, BaiduYun

Once downloaded, execute the following commands to link the datasets and weights.

git clone https://github.com/XMed-Lab/URN.git
cd URN
mkdir data
cd  data
ln -s [path to model files] models
ln -s [path to voc12] voc12
ln -s [path to coco2014] coco2014
ln -s [path to your voc pseudo-mask] voc12/VOC2012/ppmg
ln -s [path to your coco pseudo-mask] coco2014/voc_format/ppmg

Run the code

(If you don't run on server cluster based on srun, please modify the scripts "tools/dist_*.sh" refer to given scripts "tools/srun_*.sh")

Installation
cd URN
pip install mmcv==1.1.5
pip install -e .

(If you meet installation problems, please refer to mmsegmentation)

Train segmentation for the first time (you can skip it by intermediate weights)
cd URN
bash tools/slurm_train.sh [cluster partition] python configs/pspnet_wsss/pspnet_res2net_20k_voc12aug_pus.py work_dirs/voc12_r2n_pus 8
Uncertainty estimation and generate cyclic pseudo-mask
bash tools/slurm_test.sh [cluster partition] python configs/pspnet_wsss/pspnet_res2net_20k_voc12aug_uncertainty.py [intermediate weights] 8
Train segmentation with reweight strategy
bash tools/slurm_train.sh [cluster partition] python configs/pspnet_wsss/pspnet_res2net_20k_voc12aug_urn.py work_dirs/voc12_r2n_urn 8
Notes:
  1. We provide other backbones, including ResNet101, ScaleNet101, Wide-ResNet38
  2. Configs of COCO14 are provided in "configs/pspnet_wsss"
  3. It's suggested to use multiple cluster nodes to accelerate the genetation of pseudo-mask when use "tools/slurm_test.sh"
  4. Run "tools/run_pmm.sh" to get baselines of PMM

License

Please refer to: LICENSE.

Comments
  • The generation of published ''Pseduo-masks from classification phase''

    The generation of published ''Pseduo-masks from classification phase''

    Hey, thank you for your impressive works. Would you please kindly give us an explanation to the published ''Pseduo-masks from classification phase''?

    I guess it is the CAM generated from the trained classification network, and then post-processed by dCRF, finally with a threshold to distinguish background and certain class. If the procedure is correct, would you please let me know the threshold between bg and fg? Or if there is anything different, please let me know.

    Thank you so much!

    opened by Zhengyang1995 2
  • About generate mask

    About generate mask

    Thanks for your perfect work! Now I have a question. If I want to generate WSSS mask for coco dataset? Can I use the weight you provide directly? Which weight and config file, I should use? for res2net. Thanks for helping!!!

    opened by ZechengLi19 1
  • version problem

    version problem

    I would like to know what is your configuration，For example: what are the versions of the cuda,pytorch and python ? Looking forward to your reply！！！！！！

    opened by Dream-ai 1
  • MMCV version problem

    MMCV version problem

    Can I run the code with mmcv==1.2.0 in ubuntu20.04? I can't install mmcv==1.1.5 which need pytorch <= 1.6.0 and cuda <= 10.2, and I just could install cuda >= 11.0 with mmcv==1.2.0, pytorch==1.7.0+cu110. In your mmseg/init.py file, I see that the mmcv version is >=1.1.4 and <=1.2.0, but I can't success when runing in the mmcv==1.2.0 environment, I believe the version of pytorch,cuda,mmcv is compatible, but I also get the problem just like : /mmcv/_ext.cpython-37m-x86_64-linux-gnu.so: undefined symbol:_ZN6caffe28TypeMeta21_typeMetaDataInstanceIdEEPKNS_6detail12TypeMetaDataEv

    opened by LYY-HH 1
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
Owner
XMed-Lab
Medical AI and Computer Vision Group, HKUST
XMed-Lab
GitHub
Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.

WSDEC This is the official repo for our NeurIPS paper Weakly Supervised Dense Event Captioning in Videos. Description Repo directories ./: global conf

Melon(Xuguang Duan) 96 Nov 1, 2022
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Jiwoon Ahn 337 Dec 15, 2022
[CVPR 2021] Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision

TorchSemiSeg [CVPR 2021] Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision by Xiaokang Chen1, Yuhui Yuan2, Gang Zeng1, Jingdong Wang

Chen XiaoKang 387 Jan 8, 2023
[CVPR 2022] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Using Unreliable Pseudo Labels Official PyTorch implementation of Semi-Supervised Semantic Segmentation Using Unreliable Pseudo Labels, CVPR 2022. Ple

Haochen Wang 268 Dec 24, 2022
DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation

DCT-Mask: Discrete Cosine Transform Mask Representation for Instance Segmentation This project hosts the code for implementing the DCT-MASK algorithms

Alibaba Cloud 57 Nov 27, 2022
git《Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser》(2021) GitHub: [fig5]

Pseudo-ISP: Learning Pseudo In-camera Signal Processing Pipeline from A Color Image Denoiser Abstract The success of deep denoisers on real-world colo

Yue Cao 51 Nov 22, 2022
A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains (IJCV submission)

wsss-analysis The code of: A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains, arXiv pre-print 2019 paper.

Lyndon Chan 48 Dec 18, 2022
Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation

Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation The code of: Context Decoupling Augmentation for Weakly Supervised Semanti

null 54 Dec 12, 2022
Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation

Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation (AAAI 2021) Official pytorch implementation of our paper: Discriminative

Beom 74 Dec 27, 2022
Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation Input Image Initial CAM Successive Maps with adversar

Jungbeom Lee 110 Dec 7, 2022
Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

One Thing One Click One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR2021) Code for the paper One Thi

null 44 Dec 12, 2022
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

The Official PyTorch Implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Shiyi Lan 3 Oct 15, 2021
Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021)

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021) The implementation of Reducing Infromation Bottleneck for W

Jungbeom Lee 81 Dec 16, 2022
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision The PyTorch implementation of DiscoBox: Weakly Supe

Shiyi Lan 1 Oct 23, 2021
Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021)

Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021) This is the implementation of PSD (ICCV 2021),

null 12 Dec 12, 2022
Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation (CVPR 2022)

CCAM (Unsupervised) Code repository for our paper "CCAM: Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localizati

Computer Vision Insitute, SZU 113 Dec 27, 2022
[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

Ye Du 96 Dec 30, 2022
DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Differentiable Model Compression via Pseudo Quantization Noise DiffQ performs differentiable quantization using pseudo quantization noise. It can auto

Facebook Research 145 Dec 30, 2022
PyTorch implementation of SmoothGrad: removing noise by adding noise.

SmoothGrad implementation in PyTorch PyTorch implementation of SmoothGrad: removing noise by adding noise. Vanilla Gradients SmoothGrad Guided backpro

SSKH 143 Jan 5, 2023