Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021)

Related tags

Deep Learning weakly-supervised-learning rib weakly-supervised-segmentation neurips2021
Overview

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021)

The implementation of Reducing Infromation Bottleneck for Weakly Supervised Semantic Segmentation, Jungbeom Lee, Jooyoung Choi, Jisoo Mok, and Sungroh Yoon, NeurIPS 2021. [[paper]]

outline

outline

Abstract

Weakly supervised semantic segmentation produces pixel-level localization from class labels; however, a classifier trained on such labels is likely to focus on a small discriminative region of the target object. We interpret this phenomenon using the information bottleneck principle: the final layer of a deep neural network, activated by the sigmoid or softmax activation functions, causes an information bottleneck, and as a result, only a subset of the task-relevant information is passed on to the output. We first support this argument through a simulated toy experiment and then propose a method to reduce the information bottleneck by removing the last activation function. In addition, we introduce a new pooling method that further encourages the transmission of information from non-discriminative regions to the classification. Our experimental evaluations demonstrate that this simple modification significantly improves the quality of localization maps on both the PASCAL VOC 2012 and MS COCO 2014 datasets, exhibiting a new state-of-the-art performance for weakly supervised semantic segmentation.

Installation

  • We kindly refer to the offical implementation of IRN.

Usage

Step 1. Prepare Dataset

  • Download Pascal VOC dataset here.

  • Download MS COCO images from the official COCO website here.

  • Download semantic segmentation annotations for the MS COCO dataset here.

  • Directory hierarchy

    Dataset
    ├── VOC2012_SEG_AUG       # unzip VOC2012_SEG_AUG.zip           
    ├── coco_2017             # mkdir coco_2017
    │   ├── coco_seg_anno     # included in coco_annotations_semantic.zip
    └── └── JPEGImages        # include train and val images downloaded from the official COCO website

Step 2. Prepare pre-trained classifier

  • Pre-trained model used in this paper: Pascal VOC, MS COCO.
  • You can also train your own classifiers following IRN.

Step 3. Generate and evaluate the pseudo ground-truth masks for PASCAL VOC and MS COCO

  • PASCAL VOC
bash get_pseudo_gt_VOC.sh
  • MS COCO
bash get_pseudo_gt_COCO.sh

Step 4. Train a semantic segmentation network

Acknowledgment

This code is heavily borrowed from IRN, thanks jiwoon-ahn!

You might also like...
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision The PyTorch implementation of DiscoBox: Weakly Supe

Shiyi Lan 1 Oct 23, 2021
Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021)

Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021) This is the implementation of PSD (ICCV 2021),

null 12 Dec 12, 2022
Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation (CVPR 2022)
Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation (CVPR 2022)

CCAM (Unsupervised) Code repository for our paper "CCAM: Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localizati

Computer Vision Insitute, SZU 113 Dec 27, 2022
[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast
[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

Ye Du 96 Dec 30, 2022
Official PyTorch Implementation for InfoSwap: Information Bottleneck Disentanglement for Identity Swapping
Official PyTorch Implementation for InfoSwap: Information Bottleneck Disentanglement for Identity Swapping

InfoSwap: Information Bottleneck Disentanglement for Identity Swapping Code usage Please check out the user manual page. Paper Gege Gao, Huaibo Huang,

Grace Hešeri 56 Dec 20, 2022
A self-supervised 3D representation learning framework named viewpoint bottleneck.

Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck Paper Created by Liyi Luo, Beiwen Tian, Hao Zhao and Guyue Zhou from Institute for AI In

null 63 Aug 11, 2022
A self-supervised 3D representation learning framework named viewpoint bottleneck.

Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck Paper Created by Liyi Luo, Beiwen Tian, Hao Zhao and Guyue Zhou from Institute for AI In

null 42 Sep 24, 2021
Codes for TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization.
Codes for TS-CAM: Token Semantic Coupled Attention Map for Weakly Supervised Object Localization.

TS-CAM: Token Semantic Coupled Attention Map for Weakly SupervisedObject Localization This is the official implementaion of paper TS-CAM: Token Semant

vasgaowei 112 Jan 2, 2023
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

Hybrid-Supervised Object Detection System Object detection system trained by hybrid-supervision/weakly semi-supervision (HSOD/WSSOD): This project is

null 5 Dec 10, 2022
Comments
  • Performance gap on VOC

    Performance gap on VOC

    Hi, I am trying to reproduce your results on VOC, the mIoU of the final pseudo label is 67.1%, which is similar to your reported results in the paper. But when I use those generated pseudo labels to train deeplab-v2 (initializing with image-net pretrained model), the final performance with CRF can only reach 65.3%.

    I am following the same setting mentioned in your another issue, with batch size:10, iter_max: 30000, lr: 2.5e-4, dataset scale for training: [0.5, 0.75, 1.0, 1.25, 1.5, 1.75, 2.0], and the balanced CEloss.

    I want to know whether your config is changed for this paper. Or whether I missed some important settings.

    Many thanks for your great work.

    opened by zbf1991 4
  • COCO performance

    COCO performance

    Hi, I also reproduce the pseudo labels on COCO, but the final mIoU of the pseudo labels only reaches 39.9%, which is much lower than your report in your paper, and here is my log, could you please share your log file or your final pseudo labels so that I can check what is the problem?

    I directly use your code without any changes.

    Here is the main result from my Log file:

    result/cam_RIB_coco [0.2722552891237194, 0.2786577865531708, 0.2848951651555352, 0.29093880903208663, 0.2967588555960802, 0.3023511215664053, 0.30768588721569845, 0.3127349367603064, 0.3174742008072476, 0.3219033934796489, 0.32601260238173524, 0.3297762164192385, 0.33319474995831166, 0.3362583752094398, 0.3389686334201114, 0.3413257565050376, 0.34331515012800756, 0.3449353523677618, 0.3461883620849829, 0.34709014345965156, 0.34764327026979114, 0.34783290477587275, 0.34766379734497427, 0.34713057606322156, 0.3462663810632868, 0.3450816387680953, 0.3435831083609862, 0.3417649041730949, 0.33964585020688953, 0.33723733585199694, 0.33452463044941305, 0.3315106653971657, 0.3282249115321002, 0.3246616273393496, 0.32083791932265093, 0.3167660542320165, 0.31243464959554595, 0.30784246947610633, 0.3030084580507455, 0.2979410572954709, 0.2926588965608258] 0.34783290477587275

    step.eval_sem_seg: Sun Jan 2 12:33:03 2022 total images 82783 0.07819244921909785 0.15218531542532188 0.4476158785550421 0.1573540223429845 {'iou': array([0.76962224, 0.57405399, 0.45372461, 0.42301975, 0.66804454, 0.44788392, 0.68736178, 0.43598339, 0.45119599, 0.34367053, 0.16606077, 0.51578795, 0.37524895, 0.56005069, 0.34876581, 0.35976116, 0.65993155, 0.60979095, 0.58667596, 0.51476726, 0.60375777, 0.76226191, 0.61406926, 0.75800884, 0.65777299, 0.20870498, 0.58212256, 0.12488388, 0.27468946, 0.53710008, 0.57887801, 0.09641619, 0.24943139, 0.26747072, 0.30592797, 0.14781007, 0.12192337, 0.32480459, 0.14675007, 0.17803882, 0.33367101, 0.22747043, 0.29426168, 0.08872426, 0.10570477, 0.05069256, 0.30010995, 0.59472847, 0.46239021, 0.47639645, 0.57624603, 0.39865673, 0.32390225, 0.54283463, 0.61383191, 0.57325726, 0.46447526, 0.25194392, 0.46235557, 0.30174565, 0.5464725 , 0.23060425, 0.49820743, 0.43210952, 0.5080675 , 0.19334627, 0.26914282, 0.48374467, 0.49687464, 0.43566503, 0.39545811, 0.20909918, 0.24580516, 0.41792358, 0.37384429, 0.23341467, 0.29604958, 0.23709656, 0.6229902 , 0.09117136, 0.21931909]), 'miou': 0.3996546933767093}

    opened by zbf1991 2
  • Any extra saliency supervision on the COCO dataset

    Any extra saliency supervision on the COCO dataset

    Hey, this is a really awesome work! Just wonder whether you use extra saliency maps for supervision when training segmentation networks on the COCO dataset. Looking forward to your reply.

    opened by rulixiang 1
Owner
Jungbeom Lee
Jungbeom Lee
GitHub
Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.

WSDEC This is the official repo for our NeurIPS paper Weakly Supervised Dense Event Captioning in Videos. Description Repo directories ./: global conf

Melon(Xuguang Duan) 96 Nov 1, 2022
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Jiwoon Ahn 337 Dec 15, 2022
Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation Input Image Initial CAM Successive Maps with adversar

Jungbeom Lee 110 Dec 7, 2022
Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

One Thing One Click One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR2021) Code for the paper One Thi

null 44 Dec 12, 2022
Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation

Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation This paper has been accepted and early accessed

Yun Liu 39 Sep 20, 2022
The official implementation of NeurIPS 2021 paper: Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attacks

The official implementation of NeurIPS 2021 paper: Finding Optimal Tangent Points for Reducing Distortions of Hard-label Attacks

machen 11 Nov 27, 2022
A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains (IJCV submission)

wsss-analysis The code of: A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains, arXiv pre-print 2019 paper.

Lyndon Chan 48 Dec 18, 2022
Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation

Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation The code of: Context Decoupling Augmentation for Weakly Supervised Semanti

null 54 Dec 12, 2022
Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation

Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation (AAAI 2021) Official pytorch implementation of our paper: Discriminative

Beom 74 Dec 27, 2022
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

The Official PyTorch Implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Shiyi Lan 3 Oct 15, 2021