A Python library for adversarial machine learning focusing on benchmarking adversarial robustness.

Tsinghua Machine Learning Group

Last update: Dec 20, 2022

Related tags

Deep Learning nes pca bim benchmark-framework evolutionary spsa boundary adversarial-machine-learning distillation fgsm adversarial-attacks deepfool adversarial-robustness mi-fgsm mmlda hgd

Overview

ARES

This repository contains the code for ARES (Adversarial Robustness Evaluation for Safety), a Python library for adversarial machine learning research focusing on benchmarking adversarial robustness on image classification correctly and comprehensively.

We benchmark the adversarial robustness using 15 attacks and 16 defenses under complete threat models, which is described in the following paper

Benchmarking Adversarial Robustness on Image Classification (CVPR 2020, Oral)

Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, and Jun Zhu.

Feature overview:

Built on TensorFlow, and support TensorFlow & PyTorch models with the same interface.
Support many attacks in various threat models.
Provide ready-to-use pre-trained baseline models (8 on ImageNet & 8 on CIFAR10).
Provide efficient & easy-to-use tools for benchmarking models.

Citation

If you find ARES useful, you could cite our paper on benchmarking adversarial robustness using all models, all attacks & defenses supported in ARES. We provide a BibTeX entry of this paper below:

@inproceedings{dong2020benchmarking,
  title={Benchmarking Adversarial Robustness on Image Classification},
  author={Dong, Yinpeng and Fu, Qi-An and Yang, Xiao and Pang, Tianyu and Su, Hang and Xiao, Zihao and Zhu, Jun},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages={321--331},
  year={2020}
}

Installation

Since ARES is still under development, please clone the repository and install the package:

git clone https://github.com/thu-ml/ares
cd ares/
pip install -e .

The requirements.txt includes its dependencies, you might want to change PyTorch's version as well as TensorFlow 1's version. TensorFlow 1.13 or later should work fine.

As for python version, Python 3.5 or later should work fine.

The Boundary attack and the Evolutionary attack require mpi4py and a working MPI with enough localhost slots. For example, you could set the OMPI_MCA_rmaps_base_oversubscribe environment variable to yes for OpenMPI.

Download Datasets & Model Checkpoints

By default, ARES would save datasets and model checkpoints under the ~/.ares directory. You could override it by setting the ARES_RES_DIR environment variable to an alternative location.

We support 2 datasets: CIFAR-10 and ImageNet.

To download the CIFAR-10 dataset, please run:

python3 ares/dataset/cifar10.py

To download the ImageNet dataset, please run:

python3 ares/dataset/imagenet.py

for instructions.

ARES includes third party models' code in the third_party/ directory as git submodules. Before you use these models, you need to initialize these submodules:

git submodule init
git submodule update --depth 1

The example/cifar10 directory and example/imagenet directories include wrappers for these models. Run the model's .py file to download its checkpoint or view instructions for downloading. For example, if you want to download the ResNet56 model's checkpoint, please run:

python3 example/cifar10/resnet56.py

Documentation

We provide API docs as well as tutorials at https://thu-ml-ares.rtfd.io/.

Quick Examples

ARES provides command line interface to run benchmarks. For example, to run distortion benchmark on ResNet56 model for CIFAR-10 dataset using CLI:

python3 -m ares.benchmark.distortion_cli --method mim --dataset cifar10 --offset 0 --count 1000 --output mim.npy example/cifar10/resnet56.py --distortion 0.1 --goal ut --distance-metric l_inf --batch-size 100 --iteration 10 --decay-factor 1.0 --logger

This command would find the minimal adversarial distortion achieved using the MIM attack with decay factor of 1.0 on the example/cifar10/resnet56.py model with L∞ distance and save the result to mim.npy.

For more examples and usages (e.g. how to define new models), please browse our documentation website mentioned before.

Acknowledgement

This work was supported by the National Key Research and Development Program of China, Beijing Academy of Artificial Intelligence (BAAI), a grant from Tsinghua Institute for Guo Qiang.

Comments

Running test_imagenet_model.py very slowly
Hello, dear guys from thu-ml.

Thanks for your program.

I try to run test_imagenet_model.py for imagenet datasets with L∞ Attack.

The program runs very slowly.

I notice the requirement.txt set "tensorflow=1.15.4", not the "tensorflow-gpu=1.15.4".

I install with tensorflow-gpu 1.15.4 and make sure tensorflow.test.is_gpu_available() 's value TRUE.

I also notice:

config = tf.ConfigProto() config.gpu_options.allow_growth = True session = tf.Session(config=config)

My question is:

What others should I do when I run gpu program based on ImageNet.

I add adversarial attack code in: https://github.com/thu-ml/realsafe/blob/master/realsafe/dataset/imagenet.py

def _load_image(filename, to_height, to_width, clip): ''' Load image into uint8 tensor from file. ''' img = Image.open(os.path.join(PATH_IMGS, filename)) if img.mode != 'RGB': img = img.convert(mode='RGB') if clip: img = np.array(img) height, width = img.shape[0], img.shape[1] # pylint: disable=E1136 # pylint/issues/3139 center = int(0.875 * min(height, width)) offset_height, offset_width = (height - center + 1) // 2, (width - center + 1) // 2 img = img[offset_height:offset_height+center, offset_width:offset_width+center, :] img = Image.fromarray(img) return tf.convert_to_tensor(np.array(img.resize((to_height, to_width))))

Before "if clip" and after " img = img.convert(mode='RGB')" is my modification, would this modification before tensor conversion and placeholder filling slow down the operation speed?

Thanks & Regards! Momo
opened by momo1986 14
About the adversarial defense model on CIFAR-10 and ImageNet in Keras or higher tensorflow

Hello, thanks for sharing.

Is there the adversarial training model for defense available in Keras or higher tensorflow in ImageNet and CIFAR-10?

Where can I refer to it?

Thanks & Regards!

opened by momo1986 4

how to integrate other models into realsafe? Like tensorflow hub models

i want use other models to generate adversarial samples, but something wrong with it.can you tell me how to do it? This is my code

import tensorflow as tf
import tensorflow_hub as hub

from realsafe.model.base import ClassifierWithLogits
from realsafe.utils import get_res_path

# MODEL_PATH = get_res_path('./cifar10/resnet56.ckpt')


def load(session):
    model = VGGDEEP()
    # model.load(MODEL_PATH, session)
    return model

class VGGDEEP(ClassifierWithLogits):
    def __init__(self):
        ClassifierWithLogits.__init__(self,
                                      x_min=0.0, x_max=1.0, x_shape=(32, 32, 3,), x_dtype=tf.float32,
                                      y_dtype=tf.int32, n_class=10)
        self.model = hub.Module("https://hub.tensorflow.google.cn/deepmind/ganeval-cifar10-convnet/1")

    def _logits_and_labels(self, xs_ph):
        logits = self.model(xs_ph)
        predicts = tf.nn.softmax(logits)
        predicted_labels = tf.argmax(predicts, 1, output_type=tf.int32)
        return logits, predicted_labels

    def load(self, model_path, session):
        # var_list_pre = set(tf.global_variables())
        # x_input = tf.placeholder(tf.float32, shape=(None,) + self.x_shape)
        # self.model.inference(x_input, self.num_residual_blocks, reuse=tf.AUTO_REUSE)
        # var_list_post = set(tf.global_variables())
        # var_list = list(var_list_post - var_list_pre)
        # if len(var_list) > 0:
        #     saver = tf.train.Saver(var_list=var_list)
        #     saver.restore(session, model_path)
        pass

as you see ,i want to use this model(https://hub.tensorflow.google.cn/deepmind/ganeval-cifar10-convnet/1) to genernate adv samples.

opened by mayfly227 3

Get error when updating the submodules

Hello, I got an error when updating submodules. Error log:

fatal: 引用不是一个树：c6e893728c3e7355e3bc689352011a55798b0e95
无法在子模组路径 'third_party/models' 中检出 'adc01cd76ae0d9d3b2e8dde3ec6bf4086f7da046'
无法在子模组路径 'third_party/trades' 中检出 'c6e893728c3e7355e3bc689352011a55798b0e95'

Is there any solutions?

opened by Wenzhao-Xiang 2

Bump tensorflow from 1.15.4 to 2.5.3
Bumps tensorflow from 1.15.4 to 2.5.3.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.5.3

Release 2.5.3

Note: This is the last release in the 2.5 series.

This releases introduces several vulnerability fixes:

Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)

Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)

Fixes a heap OOB access in Dequantize (CVE-2022-21726)

Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)

Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)

Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)

Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)

Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)

Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)

Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)

Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)

Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)

Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)

Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)

Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)

Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)

Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)

Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)

Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)

Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)

Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)

Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)

Fixes an integer overflow in TFLite array creation (CVE-2022-23558)

Fixes an integer overflow in TFLite (CVE-2022-23559)

Fixes a dangerous OOB write in TFLite (CVE-2022-23561)

Fixes a vulnerability leading to read and write outside of bounds in TFLite (CVE-2022-23560)

Fixes a set of vulnerabilities caused by using insecure temporary files (CVE-2022-23563)

Fixes an integer overflow in Range resulting in undefined behavior and OOM (CVE-2022-23562)

Fixes a vulnerability where missing validation causes tf.sparse.split to crash when axis is a tuple (CVE-2021-41206)

Fixes a CHECK-fail when decoding resource handles from proto (CVE-2022-23564)

Fixes a CHECK-fail with repeated AttrDef (CVE-2022-23565)

Fixes a heap OOB write in Grappler (CVE-2022-23566)

Fixes a CHECK-fail when decoding invalid tensors from proto (CVE-2022-23571)

Fixes an unitialized variable access in AssignOp (CVE-2022-23573)

Fixes an integer overflow in OpLevelCostEstimator::CalculateTensorSize (CVE-2022-23575)

Fixes an integer overflow in OpLevelCostEstimator::CalculateOutputSize (CVE-2022-23576)

Fixes a null dereference in GetInitOp (CVE-2022-23577)

Fixes a memory leak when a graph node is invalid (CVE-2022-23578)

Fixes an abort caused by allocating a vector that is too large (CVE-2022-23580)

Fixes multiple CHECK-failures during Grappler's IsSimplifiableReshape (CVE-2022-23581)

Fixes multiple CHECK-failures during Grappler's SafeToRemoveIdentity (CVE-2022-23579)

Fixes multiple CHECK-failures in TensorByteSize (CVE-2022-23582)

Fixes multiple CHECK-failures in binary ops due to type confusion (CVE-2022-23583)

... (truncated)

Changelog

Sourced from tensorflow's changelog.

Release 2.5.3

This releases introduces several vulnerability fixes:

Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)

Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)

Fixes a heap OOB access in Dequantize (CVE-2022-21726)

Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)

Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)

Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)

Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)

Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)

Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)

Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)

Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)

Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)

Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)

Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)

Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)

Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)

Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)

Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)

Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)

Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)

Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)

Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)

... (truncated)

Commits

959e9b2 Merge pull request #54213 from tensorflow/fix-sanity-on-r2.5

d05fcbc Fix sanity build

f2526a0 Merge pull request #54205 from tensorflow/disable-flaky-tests-on-r2.5

a5f94df Disable flaky test

7babe52 Merge pull request #54201 from tensorflow/cherrypick-510ae18200d0a4fad797c0bf...

0e5d378 Set Env Variable to override Setuptools new behavior

fdd4195 Merge pull request #54176 from tensorflow-jenkins/relnotes-2.5.3-6805

4083165 Update RELEASE.md

a2bb7f1 Merge pull request #54185 from tensorflow/cherrypick-d437dec4d549fc30f9b85c75...

5777ea3 Update third_party/icu/workspace.bzl

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump tensorflow from 1.15.4 to 2.5.2
Bumps tensorflow from 1.15.4 to 2.5.2.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.5.2

Release 2.5.2

This release introduces several vulnerability fixes:

Fixes a code injection issue in saved_model_cli (CVE-2021-41228)

Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)

Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)

Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)

Fixes a heap OOB in SparseBinCount (CVE-2021-41226)

Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)

Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)

Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)

Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)

Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)

Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)

Fixes a heap buffer overflow in Transpose (CVE-2021-41216)

Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)

Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)

Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)

Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)

Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)

Fixes an FPE in ParallelConcat (CVE-2021-41207)

Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)

Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)

Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)

Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)

Fixes a segfault produced while copying constant resource tensor (CVE-2021-41204)

Fixes a vulnerability caused by unitialized access in EinsumHelper::ParseEquation (CVE-2021-41201)

Fixes several vulnerabilities and segfaults caused by missing validation during checkpoint loading (CVE-2021-41203)

Fixes an overflow producing a crash in tf.range (CVE-2021-41202)

Fixes an overflow producing a crash in tf.image.resize when size is large (CVE-2021-41199)

Fixes an overflow producing a crash in tf.tile when tiling tensor is large (CVE-2021-41198)

Fixes a vulnerability produced due to incomplete validation in tf.summary.create_file_writer (CVE-2021-41200)

Fixes multiple crashes due to overflow and CHECK-fail in ops with large tensor shapes (CVE-2021-41197)

Fixes a crash in max_pool3d when size argument is 0 or negative (CVE-2021-41196)

Fixes a crash in tf.math.segment_* operations (CVE-2021-41195)

Updates curl to 7.78.0 to handle CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, and CVE-2021-22926.

TensorFlow 2.5.1

Release 2.5.1

This release introduces several vulnerability fixes:

Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)

Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)

Fixes a null pointer dereference in CompressElement (CVE-2021-37637)

Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)

Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)

Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)

... (truncated)

Changelog

Sourced from tensorflow's changelog.

Release 2.5.2

This release introduces several vulnerability fixes:

Fixes a code injection issue in saved_model_cli (CVE-2021-41228)

Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)

Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)

Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)

Fixes a heap OOB in SparseBinCount (CVE-2021-41226)

Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)

Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)

Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)

Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)

Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)

Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)

Fixes a heap buffer overflow in Transpose (CVE-2021-41216)

Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)

Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)

Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)

Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)

Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)

Fixes an FPE in ParallelConcat ([CVE-2021-41207] (https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-41207))

Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)

Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)

Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)

Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)

... (truncated)

Commits

957590e Merge pull request #52873 from tensorflow-jenkins/relnotes-2.5.2-20787

2e1d16d Update RELEASE.md

2fa6dd9 Merge pull request #52877 from tensorflow-jenkins/version-numbers-2.5.2-192

4807489 Merge pull request #52881 from tensorflow/fix-build-1-on-r2.5

d398bdf Disable failing test

857ad5e Merge pull request #52878 from tensorflow/fix-build-1-on-r2.5

6c2a215 Disable failing test

f5c57d4 Update version numbers to 2.5.2

e51f949 Insert release notes place-fill

2620d2c Merge pull request #52863 from tensorflow/fix-build-3-on-r2.5

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump tensorflow from 1.15.4 to 2.5.1
Bumps tensorflow from 1.15.4 to 2.5.1.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.5.1

Release 2.5.1

This release introduces several vulnerability fixes:

Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)

Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)

Fixes a null pointer dereference in CompressElement (CVE-2021-37637)

Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)

Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)

Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)

Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)

Fixes a heap OOB in RaggedGather (CVE-2021-37641)

Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)

Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)

Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)

Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)

Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)

Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)

Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)

Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)

Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)

Fixes a use after free in boosted trees creation (CVE-2021-37652)

Fixes a division by 0 in ResourceGather (CVE-2021-37653)

Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)

Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)

Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse (CVE-2021-37656)

Fixes an undefined behavior arising from reference binding to nullptr in MatrixDiagV* ops (CVE-2021-37657)

Fixes an undefined behavior arising from reference binding to nullptr in MatrixSetDiagV* ops (CVE-2021-37658)

Fixes an undefined behavior arising from reference binding to nullptr and heap OOB in binary cwise ops (CVE-2021-37659)

Fixes a division by 0 in inplace operations (CVE-2021-37660)

Fixes a crash caused by integer conversion to unsigned (CVE-2021-37661)

Fixes an undefined behavior arising from reference binding to nullptr in boosted trees (CVE-2021-37662)

Fixes a heap OOB in boosted trees (CVE-2021-37664)

Fixes vulnerabilities arising from incomplete validation in QuantizeV2 (CVE-2021-37663)

Fixes vulnerabilities arising from incomplete validation in MKL requantization (CVE-2021-37665)

Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToVariant (CVE-2021-37666)

Fixes an undefined behavior arising from reference binding to nullptr in unicode encoding (CVE-2021-37667)

Fixes an FPE in tf.raw_ops.UnravelIndex (CVE-2021-37668)

Fixes a crash in NMS ops caused by integer conversion to unsigned (CVE-2021-37669)

Fixes a heap OOB in UpperBound and LowerBound (CVE-2021-37670)

Fixes an undefined behavior arising from reference binding to nullptr in map operations (CVE-2021-37671)

Fixes a heap OOB in SdcaOptimizerV2 (CVE-2021-37672)

Fixes a CHECK-fail in MapStage (CVE-2021-37673)

Fixes a vulnerability arising from incomplete validation in MaxPoolGrad (CVE-2021-37674)

Fixes an undefined behavior arising from reference binding to nullptr in shape inference (CVE-2021-37676)

Fixes a division by 0 in most convolution operators (CVE-2021-37675)

Fixes vulnerabilities arising from missing validation in shape inference for Dequantize (CVE-2021-37677)

Fixes an arbitrary code execution due to YAML deserialization (CVE-2021-37678)

Fixes a heap OOB in nested tf.map_fn with RaggedTensors (CVE-2021-37679)

... (truncated)

Changelog

Sourced from tensorflow's changelog.

Release 2.5.1

This release introduces several vulnerability fixes:

Fixes a heap out of bounds access in sparse reduction operations (CVE-2021-37635)

Fixes a floating point exception in SparseDenseCwiseDiv (CVE-2021-37636)

Fixes a null pointer dereference in CompressElement (CVE-2021-37637)

Fixes a null pointer dereference in RaggedTensorToTensor (CVE-2021-37638)

Fixes a null pointer dereference and a heap OOB read arising from operations restoring tensors (CVE-2021-37639)

Fixes an integer division by 0 in sparse reshaping (CVE-2021-37640)

Fixes a division by 0 in ResourceScatterDiv (CVE-2021-37642)

Fixes a heap OOB in RaggedGather (CVE-2021-37641)

Fixes a std::abort raised from TensorListReserve (CVE-2021-37644)

Fixes a null pointer dereference in MatrixDiagPartOp (CVE-2021-37643)

Fixes an integer overflow due to conversion to unsigned (CVE-2021-37645)

Fixes a bad allocation error in StringNGrams caused by integer conversion (CVE-2021-37646)

Fixes a null pointer dereference in SparseTensorSliceDataset (CVE-2021-37647)

Fixes an incorrect validation of SaveV2 inputs (CVE-2021-37648)

Fixes a null pointer dereference in UncompressElement (CVE-2021-37649)

Fixes a segfault and a heap buffer overflow in {Experimental,}DatasetToTFRecord (CVE-2021-37650)

Fixes a heap buffer overflow in FractionalAvgPoolGrad (CVE-2021-37651)

Fixes a use after free in boosted trees creation (CVE-2021-37652)

Fixes a division by 0 in ResourceGather (CVE-2021-37653)

Fixes a heap OOB and a CHECK fail in ResourceGather (CVE-2021-37654)

Fixes a heap OOB in ResourceScatterUpdate (CVE-2021-37655)

Fixes an undefined behavior arising from reference binding to nullptr in RaggedTensorToSparse

... (truncated)

Commits

8222c1c Merge pull request #51381 from tensorflow/mm-fix-r2.5-build

d584260 Disable broken/flaky test

f6c6ce3 Merge pull request #51367 from tensorflow-jenkins/version-numbers-2.5.1-17468

3ca7812 Update version numbers to 2.5.1

4fdf683 Merge pull request #51361 from tensorflow/mm-update-relnotes-on-r2.5

05fc01a Put CVE numbers for fixes in parentheses

bee1dc4 Update release notes for the new patch release

47beb4c Merge pull request #50597 from kruglov-dmitry/v2.5.0-sync-abseil-cmake-bazel

6f39597 Merge pull request #49383 from ashahab/abin-load-segfault-r2.5

0539b34 Merge pull request #48979 from liufengdb/r2.5-cherrypick

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Bump tensorflow from 1.15.4 to 2.5.0
Bumps tensorflow from 1.15.4 to 2.5.0.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.5.0

Release 2.5.0

Major Features and Improvements

Support for Python3.9 has been added.

tf.data:

tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.

tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).

tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.

Options returned by tf.data.Dataset.options() are no longer mutable.

tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().

tf.lite

Enabled the new MLIR-based quantization backend by default

The new backend is used for 8 bits full integer post-training quantization

The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)

Set experimental_new_quantizer in tf.lite.TFLiteConverter to False to disable this change

tf.keras

tf.keras.metrics.AUC now support logit predictions.

Enabled a new supported input type in Model.fit, tf.keras.utils.experimental.DatasetCreator, which takes a callable, dataset_fn. DatasetCreator is intended to work across all tf.distribute strategies, and is the only input type supported for Parameter Server strategy.

tf.distribute

tf.distribute.experimental.ParameterServerStrategy now supports training with Keras Model.fit when used with DatasetCreator.

Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.

TPU embedding support

Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.

PluggableDevice

Third-party devices can now connect to TensorFlow as plug-ins through StreamExecutor C API. and PluggableDevice interface.

Add custom ops and kernels through kernel and op registration C API.

Register custom graph optimization passes with graph optimization C API.

oneAPI Deep Neural Network Library (oneDNN) CPU performance optimizations from Intel-optimized TensorFlow are now available in the official x86-64 Linux and Windows builds.

They are off by default. Enable them by setting the environment variable TF_ENABLE_ONEDNN_OPTS=1.

We do not recommend using them in GPU systems, as they have not been sufficiently tested with GPUs yet.

TensorFlow pip packages are now built with CUDA11.2 and cuDNN 8.1.0

Breaking Changes

The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Bug Fixes and Other Changes

tf.keras:

Preprocessing layers API consistency changes:

StringLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization.

IntegerLookup added output_mode, sparse, and pad_to_max_tokens arguments with same semantics as TextVectorization. Renamed max_values, oov_value and mask_value to max_tokens, oov_token and mask_token to align with StringLookup and TextVectorization.

TextVectorization default for pad_to_max_tokens switched to False.

CategoryEncoding no longer supports adapt, IntegerLookup now supports equivalent functionality. max_tokens argument renamed to num_tokens.

Discretization added num_bins argument for learning bins boundaries through calling adapt on a dataset. Renamed bins argument to bin_boundaries for specifying bins without adapt.

Improvements to model saving/loading:

model.load_weights now accepts paths to saved models.

... (truncated)

Changelog

Sourced from tensorflow's changelog.

Release 2.5.0

Breaking Changes

The TF_CPP_MIN_VLOG_LEVEL environment variable has been renamed to to TF_CPP_MAX_VLOG_LEVEL which correctly describes its effect.

Known Caveats

Major Features and Improvements

TPU embedding support

Added profile_data_directory to EmbeddingConfigSpec in _tpu_estimator_embedding.py. This allows embedding lookup statistics gathered at runtime to be used in embedding layer partitioning decisions.

tf.keras.metrics.AUC now support logit predictions.

Creating tf.random.Generator under tf.distribute.Strategy scopes is now allowed (except for tf.distribute.experimental.CentralStorageStrategy and tf.distribute.experimental.ParameterServerStrategy). Different replicas will get different random-number streams.

tf.data:

tf.data service now supports strict round-robin reads, which is useful for synchronous training workloads where example sizes vary. With strict round robin reads, users can guarantee that consumers get similar-sized examples in the same step.

tf.data service now supports optional compression. Previously data would always be compressed, but now you can disable compression by passing compression=None to tf.data.experimental.service.distribute(...).

tf.data.Dataset.batch() now supports num_parallel_calls and deterministic arguments. num_parallel_calls is used to indicate that multiple input batches should be computed in parallel. With num_parallel_calls set, deterministic is used to indicate that outputs can be obtained in the non-deterministic order.

Options returned by tf.data.Dataset.options() are no longer mutable.

tf.data input pipelines can now be executed in debug mode, which disables any asynchrony, parallelism, or non-determinism and forces Python execution (as opposed to trace-compiled graph execution) of user-defined functions passed into transformations such as map. The debug mode can be enabled through tf.data.experimental.enable_debug_mode().

tf.lite

Enabled the new MLIR-based quantization backend by default

The new backend is used for 8 bits full integer post-training quantization

The new backend removes the redundant rescales and fixes some bugs (shared weight/bias, extremely small scales, etc)

... (truncated)

Commits

a4dfb8d Merge pull request #49124 from tensorflow/mm-cherrypick-tf-data-segfault-fix-...

2107b1d Merge pull request #49116 from tensorflow-jenkins/version-numbers-2.5.0-17609

16b8139 Update snapshot_dataset_op.cc

86a0d86 Merge pull request #49126 from geetachavan1/cherrypicks_X9ZNY

9436ae6 Merge pull request #49128 from geetachavan1/cherrypicks_D73J5

6b2bf99 Validate that a and b are proper sparse tensors

c03ad1a Ensure validation sticks in banded_triangular_solve_op

12a6ead Merge pull request #49120 from geetachavan1/cherrypicks_KJ5M9

b67f5b8 Merge pull request #49118 from geetachavan1/cherrypicks_BIDTR

a13c0ad [tf.data][cherrypick] Fix snapshot segfault when using repeat and prefecth

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
About the possibility to run on GeForce 3090 cards

Dear RealSafe team of thu-ml lab,

The realsafe code is in tensorflow version with 1.15.4, this means the CUDA version should be <=10.1.

In GeForce 3090 GPU infrasturctue environment, normally, the CUDA version should >= 11.1.

Do you have any tips to run on new GPU server, or do you recommend to run ares on 2080Ti or other older server?

Thanks & Regards! Momo

opened by momo1986 1
NotFoundError in test_imagenet_models.py

Hello, we met an error when we run "python test_imagenet_models.py"

We run "python realsafe/example/imagenet/ens4_adv_inception_v3.py" and checked out that three chekpoint files have been downloaded successfully.

But the error occurred like:

NotFoundError: Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint.

opened by Dxy-tju 1
MPI error in test_evolutionary

Hi, I met an error of MPI when I run "python test/test_evolutionary.py":

mpi4py.MPI.Exception: MPI_ERR_SPAWN: could not spawn processes

Below is the detail:

Traceback (most recent call last): File "test/test_evolutionary.py", line 69, in dists = next(g) File "/home/***/realsafe/realsafe/attack/evolutionary.py", line 110, in _batch_attack_generator args=[worker, xs_shm_file.name, xs_adv_shm_file.name, str(self.batch_size)]) File "mpi4py/MPI/Comm.pyx", line 1534, in mpi4py.MPI.Intracomm.Spawn mpi4py.MPI.Exception: MPI_ERR_SPAWN: could not spawn processes [1028GQ-240:17362] *** Process received signal *** [1028GQ-240:17362] Signal: Segmentation fault (11) [1028GQ-240:17362] Signal code: Address not mapped (1) [1028GQ-240:17362] Failing at address: 0x7f4fc075c980 Segmentation fault (core dumped)

opened by hongxin001 1
Bump numpy from 1.21.0 to 1.22.0 in /pytorch_ares
Bumps numpy from 1.21.0 to 1.22.0.

Release notes

Sourced from numpy's releases.

v1.22.0

NumPy 1.22.0 Release Notes

NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.

A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.

NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.

New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.

A new configurable allocator for use by downstream projects.

These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

Expired deprecations

Deprecated numeric style dtype strings have been removed

Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

(gh-19539)

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

(gh-19615)

... (truncated)

Commits

4adc87d Merge pull request #20685 from charris/prepare-for-1.22.0-release

fd66547 REL: Prepare for the NumPy 1.22.0 release.

125304b wip

c283859 Merge pull request #20682 from charris/backport-20416

5399c03 Merge pull request #20681 from charris/backport-20954

f9c45f8 Merge pull request #20680 from charris/backport-20663

794b36f Update armccompiler.py

d93b14e Update test_public_api.py

7662c07 Update init.py

311ab52 Update armccompiler.py

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Question about the compatibility of Pytorch

I see the overview says that ARES supports TensorFlow & PyTorch models with the same interface. but there is no direct function or interface for a trained Pytorch model to use. Does that mean user need to retrain their model with the decorator @pytorch_classifier_with_logits so that they could use the function in ARES to processing robustness testing? 尊敬的开发人员你们好，我想知道ARES是否无法直接使用我训练好的Pytorch模型进行他的鲁棒性测试？我看到我们的简介里说ARES以相同的接口支持tf和torch的模型。但是我仔细阅读我们的文档和代码后，并没有发现有类似最新版cleverhans的那种直接面向torch模型的接口。如果是我有所遗漏，请您指出我该如何操作。

opened by Ccsk-Xian 0

Error with PyTorch model

I am trying to run the WideResNet_Trades model to run the benchmark using the benchmark.iteration_cli interface. However I am getting an error shown below.

Using TensorFlow backend.
WARNING:tensorflow:From /home/itachi/repos/ares/ares/benchmark/iteration_cli.py:100: The name tf.logging.INFO is deprecated. Please use tf.compat.v1.logging.INFO instead.

WARNING:tensorflow:From /home/itachi/repos/ares/ares/benchmark/iteration_cli.py:105: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

WARNING:tensorflow:From /home/itachi/repos/ares/ares/benchmark/iteration_cli.py:107: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-05-22 01:29:30.918086: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2021-05-22 01:29:30.946135: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2799925000 Hz
2021-05-22 01:29:30.946691: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x559ebfc9e4c0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-05-22 01:29:30.946708: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2021-05-22 01:29:30.949124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-05-22 01:29:31.011989: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-22 01:29:31.012647: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x559ebfca0320 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2021-05-22 01:29:31.012678: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1060, Compute Capability 6.1
2021-05-22 01:29:31.012868: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-05-22 01:29:31.013139: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: 
name: GeForce GTX 1060 major: 6 minor: 1 memoryClockRate(GHz): 1.6705
pciBusID: 0000:01:00.0
2021-05-22 01:29:31.013309: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.0'; dlerror: libcudart.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013430: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcublas.so.10.0'; dlerror: libcublas.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013538: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcufft.so.10.0'; dlerror: libcufft.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013642: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcurand.so.10.0'; dlerror: libcurand.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013730: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusolver.so.10.0'; dlerror: libcusolver.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.013799: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcusparse.so.10.0'; dlerror: libcusparse.so.10.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /data/Softwares_Ubuntu/libs/cudnn-10.0-linux-x64-v7.4.2.24/cuda/lib64:/home/itachi/miniconda3/envs/cuda_toolkit/lib/
2021-05-22 01:29:31.016472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-05-22 01:29:31.016493: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1662] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2021-05-22 01:29:31.016511: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-05-22 01:29:31.016521: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 
2021-05-22 01:29:31.016529: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N 
WARNING:tensorflow:From /home/itachi/repos/ares/ares/attack/utils.py:8: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING:tensorflow:From /home/itachi/repos/ares/ares/dataset/utils.py:6: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use for ... in dataset:to iterate over a dataset. If using tf.estimator, return the Dataset object directly from your input function. As a last resort, you can use tf.compat.v1.data.make_one_shot_iterator(dataset).
Traceback (most recent call last):
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1365, in _do_call
    return fn(*args)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_5
	 [[{{node Variable_5/read}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/itachi/repos/ares/ares/benchmark/iteration_cli.py", line 144, in <module>
    rs = benchmark.run(dataset, logger)
  File "/home/itachi/repos/ares/ares/benchmark/iteration.py", line 256, in run
    return self._run(dataset, logger)
  File "/home/itachi/repos/ares/ares/benchmark/iteration.py", line 113, in _run_basic
    labels, dists = next(g)
  File "/home/itachi/repos/ares/ares/attack/bim.py", line 115, in _batch_attack_generator
    self._session.run(self.setup_xs, feed_dict={self.xs_ph: xs})
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 956, in run
    run_metadata_ptr)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1359, in _do_run
    run_metadata)
  File "/home/itachi/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/client/session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_5
	 [[node Variable_5/read (defined at /miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1748) ]]

Original stack trace for 'Variable_5/read':
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/repos/ares/ares/benchmark/iteration_cli.py", line 138, in <module>
    session, cw_n_points=args.cw_n_points, **kwargs)
  File "/repos/ares/ares/benchmark/iteration.py", line 61, in __init__
    self.attack = load_attack(attack_name, init_kwargs)
  File "/repos/ares/ares/benchmark/utils.py", line 37, in load_attack
    return attack_class(**kwargs)
  File "/repos/ares/ares/attack/pgd.py", line 34, in __init__
    self.rand_init_eps_var = tf.Variable(tf.zeros((self.batch_size,), dtype=self.model.x_dtype))
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 258, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 219, in _variable_v1_call
    shape=shape)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 197, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variable_scope.py", line 2519, in default_variable_creator
    shape=shape)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 262, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 1688, in __init__
    shape=shape)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 1872, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/util/dispatch.py", line 180, in wrapper
    return target(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/array_ops.py", line 203, in identity
    ret = gen_array_ops.identity(input, name=name)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_array_ops.py", line 4239, in identity
    "Identity", input=input, name=name)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py", line 794, in _apply_op_helper
    op_def=op_def)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/util/deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3357, in create_op
    attrs, op_def, compute_device)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 3426, in _create_op_internal
    op_def=op_def)
  File "/miniconda3/envs/ares_benchmark/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 1748, in __init__
    self._traceback = tf_stack.extract_stack()

Loading tensorflow session...
Loading model...
Loading dataset...
Loading attack...
Configuring attack...
Running benchmark

Command:

python3 -m ares.benchmark.iteration_cli --method pgd --dataset cifar10 --count 100 --goal ut --distance-metric l_inf --offset 0 --output ./out_logs --batch-size 2 --iteration 10 example/cifar10/wideresnet_trades.py --logger

Below is my environment details: OS: Ubuntu 20.04 Python: 3.7 Pytorch 1.5.0 Tensorflow: 1.15.4 Keras: 2.3.1

Does it have to do something with my setup or there's a bug? Please help me.

opened by sharvil10 0