Code for the CVPR2022 paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity"

Overview

Introduction

This is an official release of the paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity" (arxiv link). Overview

Abstract: Current adversarial attack research reveals the vulnerability of learning-based classifiers against carefully crafted perturbations. However, most existing attack methods have inherent limitations in cross-dataset generalization as they rely on a classification layer with a closed set of categories. Furthermore, the perturbations generated by these methods may appear in regions easily perceptible to the human visual system (HVS). To circumvent the former problem, we propose a novel algorithm that attacks semantic similarity on feature representations. In this way, we are able to fool classifiers without limiting attacks to a specific dataset. For imperceptibility, we introduce the low-frequency constraint to limit perturbations within high-frequency components, ensuring perceptual similarity between adversarial examples and originals. Extensive experiments on three datasets(CIFAR-10, CIFAR-100, and ImageNet-1K) and three public online platforms indicate that our attack can yield misleading and transferable adversarial examples across architectures and datasets. Additionally, visualization results and quantitative performance (in terms of four different metrics) show that the proposed algorithm generates more imperceptible perturbations than the state-of-the-art methods. Our code will be publicly available.

Requirements

  • python ==3.6
  • torch == 1.7.0
  • torchvision >= 0.7
  • numpy == 1.19.2
  • Pillow == 8.0.1
  • pywt

Required Dataset

  1. The data structure of Cifar10, Cifar100, ImageNet or any other datasets look like below. Please modify the dataloader at SSAH-Adversarial-master/main.py/ accordingly for your dataset structure.
/dataset/
├── Cifar10
│   │   ├── cifar-10-python.tar.gz
├── Cifar-100-python
│   │   ├── cifar-100-python.tar.gz
├── imagenet
│   ├── val
│   │   ├── n02328150

Experiments

We trained a resnet20 model with 92.6% accuracy with CIFAR1010 and a resnet20 model with 69.63% accuracy with CIFAR100. If you want to have a test, you can download our pre-trained models with the Google Drivers. If you want to use our algorithm to attack your own trained model, you can always replace our models in the file checkpoints.

(1)Attack the Models Trained on Cifar10

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/cifar10-r20.sh

(2)Attack the Models Trained on Cifar100

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/cifar100-r20.sh

(2)Attack the Models Trained on Imagenet_val

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/Imagenet_val-r50.sh

Examples

example

Results on CIFAR10 Here we offer some experiment results. You can get more results in our paper.

Name Knowledge ASR(%) L2 Linf FID LF Paper
BIM White Box 100.0 0.85 0.03 14.85 0.25 ICLR2017
PGD White Box 100.0 1.28 0.03 27.86 0.34 arxiv link
MIM White Box 100.0 1.90 0.03 26.00 0.48 CVPR2018
AutoAttack White Box 100.0 1.91 0.03 34.93 0.61 ICML2020
AdvDrop White Box 99.92 0.90 0.07 16.34 0.34 ICCV2021
C&W White Box 100.0 0.39 0.06 8.23 0.11 IEEE SSP2017
PerC-AL White Box 98.29 0.86 0.18 9.58 0.15 CVPR2020
SSA White Box 99.96 0.29 0.02 5.73 0.07 CVPR2022
SSAH White Box 99.94 0.26 0.02 5.03 0.03 CVPR2022

Citation

if the code or method help you in the research, please cite the following paper:

@article{luo2022frequency,
  title={Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity},
  author={Luo, Cheng and Lin, Qinliang and Xie, Weicheng and Wu, Bizhu and Xie, Jinheng and Shen, Linlin},
  journal={arXiv preprint arXiv:2203.05151},
  year={2022}
}
Issues
  • The variable adv was reassigned after iteration, which seems to be an error

    The variable adv was reassigned after iteration, which seems to be an error

    def forward(self, inputs: torch.Tensor) -> torch.Tensor:
    
        with torch.no_grad():
            inputs_fea = self.fea_extract(self.normalize_fn(inputs))
    
        # low frequency component
        inputs_ll = self.DWT(inputs)
        inputs_ll = self.IDWT(inputs_ll)
    
        # changes of variables
        eps = 3e-7
        modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps)
        modifier = Variable(modifier, requires_grad=True)
        modifier = modifier.to(self.device)
        optimizer = optim.Adam([modifier], lr=self.lr)
    
        lowFre_loss = nn.SmoothL1Loss(reduction='sum')
    
        for step in range(self.num_iteration):
            optimizer.zero_grad()
            self.encoder_fea.zero_grad()
    
            adv = 0.5 * (torch.tanh(modifier) + 1)
            adv_fea = self.fea_extract(self.normalize_fn(adv))
    
            adv_ll = self.DWT(adv)
            adv_ll = self.IDWT(adv_ll)
    
            pos_sim, neg_sim = self.cal_sim(adv_fea, inputs_fea)
            if step == 0:
                pos_neg_sim, indices = self.select_setp1(pos_sim, neg_sim)
            else:
                pos_neg_sim = self.select_step2(pos_sim, neg_sim, indices)
    
            sim_pos = pos_neg_sim[:, 0]
            sim_neg = pos_neg_sim[:, -1]
    
            w_p = torch.clamp_min(sim_pos.detach() - self.m, min=0)
            w_n = torch.clamp_min(1 + self.m - sim_neg.detach(), min=0)
    
            adv_cost = self.alpha * torch.sum(torch.clamp(w_p * sim_pos - w_n * sim_neg, min=0))
            lowFre_cost = self.beta * lowFre_loss(adv_ll, inputs_ll)
            total_cost = adv_cost + lowFre_cost
    
            optimizer.zero_grad()
            total_cost.backward()
            optimizer.step()
    
        adv = 0.5 * (torch.tanh(modifier.detach()) + 1)
        return adv
    
    opened by SuperCarryFu 0
  • why make modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps)?

    why make modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps)?

    previous work usually said adv = data.cuda(), so I want to know why make modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps) and adv = 0.5 * (torch.tanh(modifier) + 1)? Thank you!

    opened by wanglinourin 1
This is an official implementation of the CVPR2022 paper "Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots".

Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots Blind2Unblind Citing Blind2Unblind @inproceedings{wang2022blind2unblind, tit

demonsjin 26 May 10, 2022
CVPR2022 paper "Dense Learning based Semi-Supervised Object Detection"

[CVPR2022] DSL: Dense Learning based Semi-Supervised Object Detection DSL is the first work on Anchor-Free detector for Semi-Supervised Object Detecti

Bhchen 29 May 19, 2022
The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift

TwoStageAlign The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift Pa

Shi Guo 23 Apr 10, 2022
Official code for "Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes", CVPR2022

[CVPR 2022] Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes Dongkwon Jin, Wonhui Park, Seong-Gyun Jeong, Heeyeon Kwon, and Cha

Dongkwon Jin 55 May 18, 2022
Official code for "Towards An End-to-End Framework for Flow-Guided Video Inpainting" (CVPR2022)

E2FGVI (CVPR 2022) English | 简体中文 This repository contains the official implementation of the following paper: Towards An End-to-End Framework for Flo

Media Computing Group @ Nankai University 304 May 26, 2022
PSTR: End-to-End One-Step Person Search With Transformers (CVPR2022)

PSTR (CVPR2022) This code is an official implementation of "PSTR: End-to-End One-Step Person Search With Transformers (CVPR2022)". End-to-end one-step

Jiale Cao 12 May 18, 2022
[CVPR2022] Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos

Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos Created by Muheng Li, Lei Chen, Yueqi Duan, Zhilan Hu, Jianjiang Feng, Jie

null 29 May 13, 2022
Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022)

Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022) By Shilong Zhang*, Zhuoran Yu*, Liyang Liu*, Xinjiang Wang, Aojun Zhou,

Shilong Zhang 44 May 21, 2022
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis Multi-View Consistent Generative Adversarial Networks for 3D-aware

Xuanmeng Zhang 52 May 23, 2022
TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022)

TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022) Ziang Cao and Ziyuan Huang and Liang Pan and Shiwei Zhang and Ziwei Liu and Changhong Fu In

Intelligent Vision for Robotics in Complex Environment 74 May 25, 2022
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

Qiaole Dong 78 May 24, 2022
FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset (CVPR2022)

FaceVerse FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang

Lizhen Wang 99 May 27, 2022
Towards Implicit Text-Guided 3D Shape Generation (CVPR2022)

Towards Implicit Text-Guided 3D Shape Generation Towards Implicit Text-Guided 3D Shape Generation (CVPR2022) Code for the paper [Towards Implicit Text

null 29 May 14, 2022
Video Frame Interpolation with Transformer (CVPR2022)

VFIformer Official PyTorch implementation of our CVPR2022 paper Video Frame Interpolation with Transformer Dependencies python >= 3.8 pytorch >= 1.8.0

DV Lab 30 May 20, 2022
[CVPR2022] Representation Compensation Networks for Continual Semantic Segmentation

RCIL [CVPR2022] Representation Compensation Networks for Continual Semantic Segmentation Chang-Bin Zhang1, Jia-Wen Xiao1, Xialei Liu1, Ying-Cong Chen2

Chang-Bin Zhang 44 May 23, 2022
A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022)

A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022) https://arxiv.org/abs/2203.09388 Jianqi Ma, Zheto

MA Jianqi, shiki 59 May 16, 2022
Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022)

Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022) Junjie Ye, Changhong Fu, Guangze Zheng, Danda Pani Paudel, and Guang Chen. Uns

Intelligent Vision for Robotics in Complex Environment 78 May 23, 2022
CVPR2022 (Oral) - Rethinking Semantic Segmentation: A Prototype View

Rethinking Semantic Segmentation: A Prototype View Rethinking Semantic Segmentation: A Prototype View, Tianfei Zhou, Wenguan Wang, Ender Konukoglu and

Tianfei Zhou 126 May 26, 2022
The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory

This repository contains the software implementation of most algorithms used or developed in my research. The LaTeX and Python code for generating the

João Fonseca 3 Apr 29, 2022