Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Overview

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation

Input Image Initial CAM Successive Maps with adversarial climbing
a b c

The implementation of Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation, Jungbeom Lee, Eunji Kim, and Sungroh Yoon, CVPR 2021. [paper]

Installation

  • We kindly refer to the offical implementation of IRN.
  • This repository is tested on Ubuntu 18.04, with Python 3.6, PyTorch 1.4, pydensecrf, scipy, chaniercv, imageio, and opencv-python.

Usage

Step 1. Prepare Dataset

  • Download PASCAL VOC 2012 benchmark: Download.

Step 2. Prepare pre-trained classifier

  • Pre-trained model used in this paper: Download.
  • You can also train your own classifiers following IRN.

Step 3. Obtain the pseudo ground-truth masks for PASCAL VOC train_aug images and evaluate them

bash get_mask_quality.sh

Step 4. Train a semantic segmentation network

Acknowledgment

This code is heavily borrowed from IRN, thanks jiwoon-ahn!

Comments
  • Questions about equation (1) and (3)

    Questions about equation (1) and (3)

    Hi @jbeomlee93 I have some questions for the equations of you paper. That is: 1. For equation(1), the paper[16] uses x' = x + gradient to adversarial attack the input image (i.e., X) and finally generate the image Xadv, which is opposited to your equation (1) (i.e., x' = x - gradient ). 2. For equation(3), the paper[28] uses iter(x' = x + gradient) to represents the adversarial attack process, which is same to equation (3)(i.e., x' = x + gradient ). I think adversarial climbing should be opposited to the adversarial attack. Sincerely waiting for your reply !

    opened by YeRen123455 4
  • how to visualize the loss landscape?

    how to visualize the loss landscape?

    in the paper you mentioned and showed an example of visualizing the loss landscape. Can you please explain how to do it ? I would really appreciate it if you can share the code for that since I am new to this topic and your implementation will be super useful!

    opened by seyeeet 4
  • Performance Gap on Init Seed

    Performance Gap on Init Seed

    Hi, I ran your codes and apply all the hyper-parameters by default. But the init seed mIoU is only 48.75, which is reported as 55.6 in your paper. I wonder if i miss something? Here is my train command line: CUDA_VISIBLE_DEVICES=4,5,6,7 python run_sample.py --voc12_root Dataset/VOC2012_SEG_AUG/ --cam_out_dir result/cam_res50 --sem_seg_out_dir result/seg_res50 --ir_label_out_dir result/ir_label_res50 --irn_weights_name sess/res50_irn.pth --make_ins_seg_pass False --eval_ins_seg_pass False

    And here is my init seed result:

    image

    opened by PlumedSerpent 2
  • About IRN

    About IRN

    Thanks for the great work. I have a question about IRN. Have you reproduced the performance of IRN? I run the code of IRN, but the results are not as good as the paper of IRN. In your implementation, will it affect the performance of AdvCAM?

    Thank you in advance for your help.

    opened by scott870430 2
  • FileNotFoundError: [Errno 2] No such file or directory: 'result/cam_adv_mask/2007_000032.npy'

    FileNotFoundError: [Errno 2] No such file or directory: 'result/cam_adv_mask/2007_000032.npy'

    Thanks for sharing your code. after downloading all the materials now I am running the bash get_mask_quality.sh but it is super slow and I am wondering how long that process should take? and what does that _work function in obtain_CAM_masking.py really do? do I need to do it?

    opened by seyeeet 2
  • Question about Table 4.

    Question about Table 4.

    Thanks for sharing this great work!

    Just wondering if you could also share the code that reproduces table 4, especially AdvCAM with SEAM.

    I found the SEAM GitHub and I assume that your AdvCAM should be replaced with cam in SEAM code as shown in the below link [https://github.com/YudeWang/SEAM/blob/c55601649c5fa676836d3ec70ec044541b7d1d83/network/resnet38_SEAM.py#L32]

    Could you please guide me on how to merge your AdvCAM into the SEAM module?

    Thanks

    opened by hchoi71 1
  • Code for coco and weakly supervised object localization

    Code for coco and weakly supervised object localization

    Hi, Thanks for your code and paper! Congratulations on your paper being accepted by TPAMI! I notice that in the version of TPAMI, additional experimental results on MS COCO dataset and ImageNet1k are presented. May I trouble you to release these code and fine-turned models? Thanks!

    opened by Jiany-Zhang 0
  • Some questions about released code

    Some questions about released code

    @jbeomlee93 ! Sorry for disturbing you again. I still have two questions about the released code. (1) In obtain_CAM_masking_super_pixel.py. Since you have used grad-cam to generate the class activation map(i.e., CAM), why don't you use resnet50.py with grad-cam to generate outputs. Actually, you used resnet50_cam.py with grad-cam to generate the outputs.

    (2) Can you share the code of "SEAM+AdvCAM" with me. I try to reproduce it by myself but the performance is not good as yours. My email address is [email protected]

    opened by YeRen123455 6
  • the result of obtian_cam_masking.py and the pseudo mask

    the result of obtian_cam_masking.py and the pseudo mask

    Hiļ¼Œthanks for your sharing!But i still have some questions look forward to your answer! is the result of the obtian_cam_masking.py a pseudo mask dirtectly used in training segmentation network?if not, can you tell how to generate it? Thank you!

    opened by whiterAutumn 4
  • obtain_CAM_masking.py takes a lot of time?

    obtain_CAM_masking.py takes a lot of time?

    Hi, your work is superb. can i know whether obtain_CAM_masking.py takes a lot of time to gets completed ? it seems there are two loops in GradCAM process. i have 103 classes.

    opened by Dhanujagithub 2
  • Semi-supervised learning setting

    Semi-supervised learning setting

    Hi thanks for sharing a great work!

    In the previous issue, you said you used the CCT's code for semi-supervised learning with the replacement of pseudo labels. Does that mean you used the default configuration provided in CCT?

    opened by won-bae 7
Owner
Jungbeom Lee
Jungbeom Lee
Code for the paper One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation, CVPR 2021.

One Thing One Click One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation (CVPR2021) Code for the paper One Thi

null 44 Dec 12, 2022
Weakly Supervised Dense Event Captioning in Videos, i.e. generating multiple sentence descriptions for a video in a weakly-supervised manner.

WSDEC This is the official repo for our NeurIPS paper Weakly Supervised Dense Event Captioning in Videos. Description Repo directories ./: global conf

Melon(Xuguang Duan) 96 Nov 1, 2022
Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation (CVPR 2022)

CCAM (Unsupervised) Code repository for our paper "CCAM: Contrastive learning of Class-agnostic Activation Map for Weakly Supervised Object Localizati

Computer Vision Insitute, SZU 113 Dec 27, 2022
[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

Ye Du 96 Dec 30, 2022
Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CVPR 2021)

Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CAC) Xin Lai*, Zhuotao Tian*, Li Jiang, Shu Liu, Hengshuang Zhao, Li

Jia Research Lab 137 Dec 14, 2022
Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CVPR 2021)

Semi-supervised Semantic Segmentation with Directional Context-aware Consistency (CAC) Xin Lai*, Zhuotao Tian*, Li Jiang, Shu Liu, Hengshuang Zhao, Li

DV Lab 137 Dec 14, 2022
[CVPR 2021] Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision

TorchSemiSeg [CVPR 2021] Semi-Supervised Semantic Segmentation with Cross Pseudo Supervision by Xiaokang Chen1, Yuhui Yuan2, Gang Zeng1, Jingdong Wang

Chen XiaoKang 387 Jan 8, 2023
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

Hybrid-Supervised Object Detection System Object detection system trained by hybrid-supervision/weakly semi-supervision (HSOD/WSSOD): This project is

null 5 Dec 10, 2022
Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021)

Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation (NeurIPS 2021) The implementation of Reducing Infromation Bottleneck for W

Jungbeom Lee 81 Dec 16, 2022
[CVPR 2022] Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

Using Unreliable Pseudo Labels Official PyTorch implementation of Semi-Supervised Semantic Segmentation Using Unreliable Pseudo Labels, CVPR 2022. Ple

Haochen Wang 268 Dec 24, 2022
CAUSE: Causality from AttribUtions on Sequence of Events

CAUSE: Causality from AttribUtions on Sequence of Events

Wei Zhang 21 Dec 1, 2022
Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

USDAN The implementation of Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, which is accepte

null 11 Nov 3, 2022
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

The Official PyTorch Implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Shiyi Lan 3 Oct 15, 2021
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision The PyTorch implementation of DiscoBox: Weakly Supe

Shiyi Lan 1 Oct 23, 2021
A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains (IJCV submission)

wsss-analysis The code of: A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains, arXiv pre-print 2019 paper.

Lyndon Chan 48 Dec 18, 2022
Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation

Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation The code of: Context Decoupling Augmentation for Weakly Supervised Semanti

null 54 Dec 12, 2022
Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation

Discriminative Region Suppression for Weakly-Supervised Semantic Segmentation (AAAI 2021) Official pytorch implementation of our paper: Discriminative

Beom 74 Dec 27, 2022
Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021)

Perturbed Self-Distillation: Weakly Supervised Large-Scale Point Cloud Semantic Segmentation (ICCV2021) This is the implementation of PSD (ICCV 2021),

null 12 Dec 12, 2022
[CVPR 2021] MiVOS - Mask Propagation module. Reproduced STM (and better) with training code :star2:. Semi-supervised video object segmentation evaluation.

MiVOS (CVPR 2021) - Mask Propagation Ho Kei Cheng, Yu-Wing Tai, Chi-Keung Tang [arXiv] [Paper PDF] [Project Page] [Papers with Code] This repo impleme

Rex Cheng 106 Jan 3, 2023