Code for the CVPR2022 paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity"

Overview

Introduction

This is an official release of the paper "Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity" (arxiv link). Overview

Abstract: Current adversarial attack research reveals the vulnerability of learning-based classifiers against carefully crafted perturbations. However, most existing attack methods have inherent limitations in cross-dataset generalization as they rely on a classification layer with a closed set of categories. Furthermore, the perturbations generated by these methods may appear in regions easily perceptible to the human visual system (HVS). To circumvent the former problem, we propose a novel algorithm that attacks semantic similarity on feature representations. In this way, we are able to fool classifiers without limiting attacks to a specific dataset. For imperceptibility, we introduce the low-frequency constraint to limit perturbations within high-frequency components, ensuring perceptual similarity between adversarial examples and originals. Extensive experiments on three datasets(CIFAR-10, CIFAR-100, and ImageNet-1K) and three public online platforms indicate that our attack can yield misleading and transferable adversarial examples across architectures and datasets. Additionally, visualization results and quantitative performance (in terms of four different metrics) show that the proposed algorithm generates more imperceptible perturbations than the state-of-the-art methods. Our code will be publicly available.

Requirements

  • python ==3.6
  • torch == 1.7.0
  • torchvision >= 0.7
  • numpy == 1.19.2
  • Pillow == 8.0.1
  • pywt

Required Dataset

  1. The data structure of Cifar10, Cifar100, ImageNet or any other datasets look like below. Please modify the dataloader at SSAH-Adversarial-master/main.py/ accordingly for your dataset structure.
/dataset/
├── Cifar10
│   │   ├── cifar-10-python.tar.gz
├── Cifar-100-python
│   │   ├── cifar-100-python.tar.gz
├── imagenet
│   ├── val
│   │   ├── n02328150

Experiments

We trained a resnet20 model with 92.6% accuracy with CIFAR1010 and a resnet20 model with 69.63% accuracy with CIFAR100. If you want to have a test, you can download our pre-trained models with the Google Drivers. If you want to use our algorithm to attack your own trained model, you can always replace our models in the file checkpoints.

(1)Attack the Models Trained on Cifar10

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/cifar10-r20.sh

(2)Attack the Models Trained on Cifar100

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/cifar100-r20.sh

(2)Attack the Models Trained on Imagenet_val

CUDA_VISIBLE_DEVICES=0,1 bash scripts/cifar/Imagenet_val-r50.sh

Examples

example

Results on CIFAR10 Here we offer some experiment results. You can get more results in our paper.

Name Knowledge ASR(%) L2 Linf FID LF Paper
BIM White Box 100.0 0.85 0.03 14.85 0.25 ICLR2017
PGD White Box 100.0 1.28 0.03 27.86 0.34 arxiv link
MIM White Box 100.0 1.90 0.03 26.00 0.48 CVPR2018
AutoAttack White Box 100.0 1.91 0.03 34.93 0.61 ICML2020
AdvDrop White Box 99.92 0.90 0.07 16.34 0.34 ICCV2021
C&W White Box 100.0 0.39 0.06 8.23 0.11 IEEE SSP2017
PerC-AL White Box 98.29 0.86 0.18 9.58 0.15 CVPR2020
SSA White Box 99.96 0.29 0.02 5.73 0.07 CVPR2022
SSAH White Box 99.94 0.26 0.02 5.03 0.03 CVPR2022

Citation

if the code or method help you in the research, please cite the following paper:

@article{luo2022frequency,
  title={Frequency-driven Imperceptible Adversarial Attack on Semantic Similarity},
  author={Luo, Cheng and Lin, Qinliang and Xie, Weicheng and Wu, Bizhu and Xie, Jinheng and Shen, Linlin},
  journal={arXiv preprint arXiv:2203.05151},
  year={2022}
}
Comments
  • the original author of dwt code

    the original author of dwt code

    Hi, thanks for sharing, I am looking for the original author of DWT.py, I tried DWT code years ago but I couldn't find the source, may I ask the original GitHub repo if you remember where it is from? thanks a lot.

    opened by ghost 1
  • Corrupted pretrained model

    Corrupted pretrained model

    I downloaded the pertained model and ran the code, but it seems that the pretrained model is corrected due to the following error:

    Traceback (most recent call last):
      File "/home/code/SSAH-adversarial-attack/main.py", line 156, in <module>
        fid = return_fid(benign_img, adv_img)
      File "/home/code/SSAH-adversarial-attack/utils/fid_score.py", line 256, in return_fid
        dims=2048)
      File "/home/code/SSAH-adversarial-attack/utils/fid_score.py", line 239, in calculate_fid_given_paths
        model = InceptionV3([block_idx]).to(device)
      File "/home/code/SSAH-adversarial-attack/utils/inception.py", line 82, in __init__
        inception = fid_inception_v3()
      File "/home/code/SSAH-adversarial-attack/utils/inception.py", line 208, in fid_inception_v3
        state_dict = torch.load(os.path.join("checkpoints", "pt_inception-2015-12-05-6726825d.pth"))
      File "/home/anaconda3/envs/ssah/lib/python3.6/site-packages/torch/serialization.py", line 595, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "/home/anaconda3/envs/ssah/lib/python3.6/site-packages/torch/serialization.py", line 781, in _legacy_load
        deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
    RuntimeError: unexpected EOF, expected 559999 more bytes. The file might be corrupted.
    
    opened by kashiani 1
  • why make modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps)?

    why make modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps)?

    previous work usually said adv = data.cuda(), so I want to know why make modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps) and adv = 0.5 * (torch.tanh(modifier) + 1)? Thank you!

    opened by wanglinourin 1
  • The variable adv was reassigned after iteration, which seems to be an error

    The variable adv was reassigned after iteration, which seems to be an error

    def forward(self, inputs: torch.Tensor) -> torch.Tensor:
    
        with torch.no_grad():
            inputs_fea = self.fea_extract(self.normalize_fn(inputs))
    
        # low frequency component
        inputs_ll = self.DWT(inputs)
        inputs_ll = self.IDWT(inputs_ll)
    
        # changes of variables
        eps = 3e-7
        modifier = torch.arctanh(inputs * (2 - eps * 2) - 1 + eps)
        modifier = Variable(modifier, requires_grad=True)
        modifier = modifier.to(self.device)
        optimizer = optim.Adam([modifier], lr=self.lr)
    
        lowFre_loss = nn.SmoothL1Loss(reduction='sum')
    
        for step in range(self.num_iteration):
            optimizer.zero_grad()
            self.encoder_fea.zero_grad()
    
            adv = 0.5 * (torch.tanh(modifier) + 1)
            adv_fea = self.fea_extract(self.normalize_fn(adv))
    
            adv_ll = self.DWT(adv)
            adv_ll = self.IDWT(adv_ll)
    
            pos_sim, neg_sim = self.cal_sim(adv_fea, inputs_fea)
            if step == 0:
                pos_neg_sim, indices = self.select_setp1(pos_sim, neg_sim)
            else:
                pos_neg_sim = self.select_step2(pos_sim, neg_sim, indices)
    
            sim_pos = pos_neg_sim[:, 0]
            sim_neg = pos_neg_sim[:, -1]
    
            w_p = torch.clamp_min(sim_pos.detach() - self.m, min=0)
            w_n = torch.clamp_min(1 + self.m - sim_neg.detach(), min=0)
    
            adv_cost = self.alpha * torch.sum(torch.clamp(w_p * sim_pos - w_n * sim_neg, min=0))
            lowFre_cost = self.beta * lowFre_loss(adv_ll, inputs_ll)
            total_cost = adv_cost + lowFre_cost
    
            optimizer.zero_grad()
            total_cost.backward()
            optimizer.step()
    
        adv = 0.5 * (torch.tanh(modifier.detach()) + 1)
        return adv
    
    opened by SuperCarryFu 0
  • Training method for ResNet20 on CIFAR dataset

    Training method for ResNet20 on CIFAR dataset

    Hi! Since you said that 'We trained a resnet20 model with 92.6% accuracy with CIFAR1010 and a resnet20 model with 69.63% accuracy with CIFAR100', I would like to know if there is any relevant training algorithm or code, and I want to train a new resnetxx model according to your method. Thanks!

    opened by YJiaaaaa 0
Owner
Leo
Leo
Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

?? Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) ?? If DaGAN is helpful in your photos/projects, please hel

Fa-Ting Hong 503 Jan 4, 2023
This is an official implementation of the CVPR2022 paper "Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots".

Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots Blind2Unblind Citing Blind2Unblind @inproceedings{wang2022blind2unblind, tit

demonsjin 58 Dec 6, 2022
CVPR2022 paper "Dense Learning based Semi-Supervised Object Detection"

[CVPR2022] DSL: Dense Learning based Semi-Supervised Object Detection DSL is the first work on Anchor-Free detector for Semi-Supervised Object Detecti

Bhchen 69 Dec 8, 2022
The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift

TwoStageAlign The official codes of our CVPR2022 paper: A Differentiable Two-stage Alignment Scheme for Burst Image Reconstruction with Large Shift Pa

Shi Guo 32 Dec 15, 2022
Official code for "Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes", CVPR2022

[CVPR 2022] Eigenlanes: Data-Driven Lane Descriptors for Structurally Diverse Lanes Dongkwon Jin, Wonhui Park, Seong-Gyun Jeong, Heeyeon Kwon, and Cha

Dongkwon Jin 106 Dec 29, 2022
Official code for "Towards An End-to-End Framework for Flow-Guided Video Inpainting" (CVPR2022)

E2FGVI (CVPR 2022) English | 简体中文 This repository contains the official implementation of the following paper: Towards An End-to-End Framework for Flo

Media Computing Group @ Nankai University 537 Jan 7, 2023
PSTR: End-to-End One-Step Person Search With Transformers (CVPR2022)

PSTR (CVPR2022) This code is an official implementation of "PSTR: End-to-End One-Step Person Search With Transformers (CVPR2022)". End-to-end one-step

Jiale Cao 28 Dec 13, 2022
[CVPR2022] Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos

Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videos Created by Muheng Li, Lei Chen, Yueqi Duan, Zhilan Hu, Jianjiang Feng, Jie

null 58 Dec 23, 2022
Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022)

Group R-CNN for Point-based Weakly Semi-supervised Object Detection (CVPR2022) By Shilong Zhang*, Zhuoran Yu*, Liyang Liu*, Xinjiang Wang, Aojun Zhou,

Shilong Zhang 129 Dec 24, 2022
Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis (CVPR2022)

Multi-View Consistent Generative Adversarial Networks for 3D-aware Image Synthesis Multi-View Consistent Generative Adversarial Networks for 3D-aware

Xuanmeng Zhang 78 Dec 10, 2022
TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022)

TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022) Ziang Cao and Ziyuan Huang and Liang Pan and Shiwei Zhang and Ziwei Liu and Changhong Fu In

Intelligent Vision for Robotics in Complex Environment 100 Dec 19, 2022
Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding (CVPR2022)

Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encoding by Qiaole Dong*, Chenjie Cao*, Yanwei Fu Paper and Supple

Qiaole Dong 190 Dec 27, 2022
FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset (CVPR2022)

FaceVerse FaceVerse: a Fine-grained and Detail-controllable 3D Face Morphable Model from a Hybrid Dataset Lizhen Wang, Zhiyuan Chen, Tao Yu, Chenguang

Lizhen Wang 219 Dec 28, 2022
Towards Implicit Text-Guided 3D Shape Generation (CVPR2022)

Towards Implicit Text-Guided 3D Shape Generation Towards Implicit Text-Guided 3D Shape Generation (CVPR2022) Code for the paper [Towards Implicit Text

null 55 Dec 16, 2022
Video Frame Interpolation with Transformer (CVPR2022)

VFIformer Official PyTorch implementation of our CVPR2022 paper Video Frame Interpolation with Transformer Dependencies python >= 3.8 pytorch >= 1.8.0

DV Lab 63 Dec 16, 2022
[CVPR2022] Representation Compensation Networks for Continual Semantic Segmentation

RCIL [CVPR2022] Representation Compensation Networks for Continual Semantic Segmentation Chang-Bin Zhang1, Jia-Wen Xiao1, Xialei Liu1, Ying-Cong Chen2

Chang-Bin Zhang 71 Dec 28, 2022
A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022)

A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution (CVPR2022) https://arxiv.org/abs/2203.09388 Jianqi Ma, Zheto

MA Jianqi, shiki 104 Jan 5, 2023
Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022)

Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022) Junjie Ye, Changhong Fu, Guangze Zheng, Danda Pani Paudel, and Guang Chen. Uns

Intelligent Vision for Robotics in Complex Environment 91 Dec 30, 2022
CVPR2022 (Oral) - Rethinking Semantic Segmentation: A Prototype View

Rethinking Semantic Segmentation: A Prototype View Rethinking Semantic Segmentation: A Prototype View, Tianfei Zhou, Wenguan Wang, Ender Konukoglu and

Tianfei Zhou 239 Dec 26, 2022