ReAct: Out-of-distribution Detection With Rectified Activations

Related tags

Deep Learning react
Overview

ReAct: Out-of-distribution Detection With Rectified Activations

This is the source code for paper ReAct: Out-of-distribution Detection With Rectified Activations by Yiyou Sun, Chuan Guo and Yixuan Li.

In this work, we propose ReAct—a simple technique for reducing model overconfidence on OOD data. Our method is motivated by novel analysis on internal activations of neural networks, which displays highly distinctive signature patterns for most OOD distributions.

Usage

1. Dataset Preparation

In-distribution dataset

Please download ImageNet-1k and place the training data and validation data in ./datasets/id_data/ILSVRC-2012/train and ./datasets/id_data/ILSVRC-2012/val, respectively.

Out-of-distribution dataset

We have curated 4 OOD datasets from iNaturalist, SUN, Places, and Textures, and de-duplicated concepts overlapped with ImageNet-1k.

For iNaturalist, SUN, and Places, we have sampled 10,000 images from the selected concepts for each dataset, which can be download via the following links:

wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/iNaturalist.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/SUN.tar.gz
wget http://pages.cs.wisc.edu/~huangrui/imagenet_ood_dataset/Places.tar.gz

For Textures, we use the entire dataset, which can be downloaded from their original website.

Please put all downloaded OOD datasets into ./datasets/ood_data/.

2. Pre-trained Model Preparation

The model we used in the paper is the pre-trained ResNet-50 and MobileNet-v2 provided by Pytorch. The download process will start upon running.

3. OOD Detection Evaluation

To reproduce our results on ResNet-50, please run:

python eval.py --threshold 1.0

To reproduce baseline approaches (Energy Score), please run:

python eval.py --threshold 1e6  #we set the threshold close to infinity, so it is the original energy score.

OOD Detection Results

ReACT achieves state-of-the-art performance averaged on the 4 OOD datasets.

results

Citation

If you use our codebase, please cite our work:

@inproceedings{sun2021react,
  title={ReAct: Out-of-distribution Detection With Rectified Activations},
  author={Sun, Yiyou and Guo, Chuan and Li, Yixuan},
  booktitle={Advances in Neural Information Processing Systems},
  year={2021}
}
You might also like...
Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

ood-text-emnlp Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them" Files fine_tune.py is used to finetune the GPT-2 mo

Udit Arora 19 Oct 28, 2022
Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples / ICLR 2018

Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples This project is for the paper "Training Confidence-Calibrated Clas

null 168 Nov 29, 2022
Codebase for Amodal Segmentation through Out-of-Task andOut-of-Distribution Generalization with a Bayesian Model

Codebase for Amodal Segmentation through Out-of-Task andOut-of-Distribution Generalization with a Bayesian Model

Yihong Sun 12 Nov 15, 2022
Categorical Depth Distribution Network for Monocular 3D Object Detection
Categorical Depth Distribution Network for Monocular 3D Object Detection

CaDDN CaDDN is a monocular-based 3D object detection method. This repository is based off of [OpenPCDet]. Categorical Depth Distribution Network for M

Toronto Robotics and AI Laboratory 289 Jan 5, 2023
Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker
Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker

Example Of Fine-Tuning BERT For Named-Entity Recognition Task And Preparing For Cloud Deployment Using Flask, React, And Docker This repository contai

Nikita 12 Dec 14, 2022
Simulator for FRC 2022 challenge: Rapid React

rrsim Simulator for FRC 2022 challenge: Rapid React out-1.mp4 Usage In order to run the simulator use the following: python3 rrsim.py [config_path] wh

null 1 Jan 18, 2022
LineBoard - Python+React+MySQL-白板即時系統改善人群行為
LineBoard - Python+React+MySQL-白板即時系統改善人群行為

LineBoard-白板即時系統改善人群行為 即時顯示實驗室的使用狀況，並遠端預約排隊，以此來改善人們的工作效率 程式架構 運作流程 使用者先至該實驗室網站預約

Bo-Jyun Huang 1 Feb 22, 2022
Notes taking website build with Docker + Django + React.
Notes taking website build with Docker + Django + React.

Notes website. Try it in browser! / But how to run? Description. This is monorepository with notes website. Website provides web interface for creatin

Kirill Zhosul 2 Jul 27, 2022
Industrial knn-based anomaly detection for images. Visit streamlit link to check out the demo.
Industrial knn-based anomaly detection for images. Visit streamlit link to check out the demo.

Industrial KNN-based Anomaly Detection ⭐ Now has streamlit support! ⭐ Run $ streamlit run streamlit_app.py This repo aims to reproduce the results of

aventau 102 Dec 26, 2022
Comments
  • A question about React menthod

    A question about React menthod

    Hello. Thanks for open-sourcing the code. I am very interested in your paper. I have a question. Your code is a combination of ReAct and other OOD detection methods. If I want to simply use the ReAct method for OOD detection, can I directly use the logit value after activation truncated as the score? logits = forward_threshold(inputs, model)

    opened by jimo17 4
  • Code to Compute the Forward Threshold

    Code to Compute the Forward Threshold

    Thank you for open-sourcing the code of this great work! I was wondering if you could also release the code that computes the threshold based on p-th percentile activation.

    opened by superctj 3
  • Question about reproducing results on CIFAR

    Question about reproducing results on CIFAR

    Hi,

    Thank you so much for opening the source code!

    I’m trying to reproduce your results on CIFAR using ResNet18 according to your paper. Since there is no checkpoints in the repo, I trained a ResNet18 on CIFAR10 by myself and its IND acc is 93.31%. I used this model to do OOD detection on CIFAR10 - SVHN, CIFAR10 - LSUN Crop, CIFAR10 - LSUN Resize, but failed to reproduce the results in your paper.

    Specifically, I ran the following commands to do the experiment,

    python compute_threshold.py --in-dataset CIFAR-10 --model-arch resnet-18
python eval.py --threshold {result computed by compute_threshold.py} --in-dataset CIFAR-10 --out-datasets SVHN LSUN LSUN_resize

    I thought the problem may lie in the process of training ResNet18, so I retrained the model strictly following the experimental details in the paper, i.e. ,

    For both CIFAR-10 and CIFAR-100, the models are trained for 100 epochs. The start learning rate is 0.1 and decays by a factor of 10 at epochs 50, 75, and 90.

    In my training, I set batch_size=1024 and used Adam as the optimizer.

    The ResNet18 I got on CIFAR10 has an IND accuracy of 90.47% but the ResNet18 trained on CIFAR100 failed to converge which only has an IND accuracy of 1% (I think lr=0.1 at the beginning is too large for CIFAR100 training, people usually choose a smaller lr to train it). Also, using the new ResNet18 trained on CIFAR10, compute_threshold.py returns 0.0 which makes me even more confused.

    I'm wondering if I did anything wrong or misunderstood something. It'll be extremely helpful if you could give me some guidance on how to reproduce the results on CIFAR.

    Thanks for your time and help!

    opened by shaoyijia 1
  • Where is mahalanobis_hyperparams

    Where is mahalanobis_hyperparams

    Hi,

    Thanks for your excellent work. When I check the code in eval.py, I just wonder where the file of "mahalanobis_hyperparams" locates. I would appreciate it if you could point it out for me.

    opened by syf-fgnb 1
Owner
CS Research Group led by Prof. Sharon Li
null
GitHub
With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified linear unit (ReLU) activation function

With this package, you can generate mixed-integer linear programming (MIP) models of trained artificial neural networks (ANNs) using the rectified linear unit (ReLU) activation function. At the moment, only TensorFlow sequential models are supported. Interfaces to either the Pyomo or Gurobi modeling environments are offered.

ChemEngAI 40 Dec 27, 2022
Official implementation for Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020

Likelihood-Regret Official implementation of Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder at NeurIPS 2020. T

Xavier 33 Oct 12, 2022
Outlier Exposure with Confidence Control for Out-of-Distribution Detection

OOD-detection-using-OECC This repository contains the essential code for the paper Outlier Exposure with Confidence Control for Out-of-Distribution De

Nazim Shaikh 64 Nov 2, 2022
Principled Detection of Out-of-Distribution Examples in Neural Networks

ODIN: Out-of-Distribution Detector for Neural Networks This is a PyTorch implementation for detecting out-of-distribution examples in neural networks.

null 189 Nov 29, 2022
The Official Implementation of the ICCV-2021 Paper: Semantically Coherent Out-of-Distribution Detection.

SCOOD-UDG (ICCV 2021) This repository is the official implementation of the paper: Semantically Coherent Out-of-Distribution Detection Jingkang Yang,

Jake YANG 62 Nov 21, 2022
Code for EMNLP 2021 paper Contrastive Out-of-Distribution Detection for Pretrained Transformers.

Contra-OOD Code for EMNLP 2021 paper Contrastive Out-of-Distribution Detection for Pretrained Transformers. Requirements PyTorch Transformers datasets

Wenxuan Zhou 27 Oct 28, 2022
Learning Confidence for Out-of-Distribution Detection in Neural Networks

Learning Confidence Estimates for Neural Networks This repository contains the code for the paper Learning Confidence for Out-of-Distribution Detectio

null 235 Jan 5, 2023
RODD: A Self-Supervised Approach for Robust Out-of-Distribution Detection

RODD Official Implementation of 2022 CVPRW Paper RODD: A Self-Supervised Approach for Robust Out-of-Distribution Detection Introduction: Recent studie

Umar Khalid 17 Oct 11, 2022
Official PyTorch implementation of the Fishr regularization for out-of-distribution generalization

Fishr: Invariant Gradient Variances for Out-of-distribution Generalization Official PyTorch implementation of the Fishr regularization for out-of-dist

null 62 Dec 22, 2022
Official repository for CVPR21 paper "Deep Stable Learning for Out-Of-Distribution Generalization".

StableNet StableNet is a deep stable learning method for out-of-distribution generalization. This is the official repo for CVPR21 paper "Deep Stable L

null 120 Dec 28, 2022