Out-of-distribution detection using the pNML regret. NeurIPS2021

Koby Bibas

Last update: Dec 2, 2022

Related tags

Deep Learning python deep-learning pytorch ood pytorch-lightning ood-detection neurips-2021

Overview

OOD Detection

Load conda environment

conda env create -f environment.yml

or install requirements:

while read requirement; do conda install --yes $requirement || pip install $requirement; done < requirements.txt

# Download OOD data
cd bash_scripts
chmod 777 ./download_data.sh
./download_data.sh

# Download pretrained models
chmod 777 ./download_models.sh
./download_models.sh

Download imagenet30

Follow https://github.com/alinlab/CSI

Imagenet30 training set: https://drive.google.com/file/d/1B5c39Fc3haOPzlehzmpTLz6xLtGyKEy4/view

Imagenet30 testing set: https://drive.google.com/file/d/13xzVuQMEhSnBRZr-YaaO08coLU2dxAUq/view

Put and untar under ./data/Imagenet30

.
├── README.md
├── data
│   ├── Imagenet30
│   │   ├── one_class_test
│   │   ├── one_class_test.tar
│   │   ├── one_class_train
│   │   └── one_class_train.tar

Execute methods

Using the pretrained models, score ood detection

cd bash_scripts
chmod 777 ./execute_methods.sh
./execute_methods.sh

Create paper's tables

cd src
python main main_create_tables.py

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (NeurIPS2021 Benchmark and Dataset Track)

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

174 Dec 22, 2022

Deep Markov Factor Analysis (NeurIPS2021)

Deep Markov Factor Analysis (DMFA) Codes and experiments for deep Markov factor analysis (DMFA) model accepted for publication at NeurIPS2021: A. Farn

2 Dec 16, 2022

[NeurIPS2021] Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks Code for NeurIPS 2021 Paper "Exploring Architectural Ingredients of A

26 Dec 1, 2022

This codebase is the official implementation of Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization (NeurIPS2021, Spotlight)

Test-Time Classifier Adjustment Module for Model-Agnostic Domain Generalization This codebase is the official implementation of Test-Time Classifier A

47 Dec 28, 2022

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021)

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021) Overview Prerequisites Linux Pytho

34 Mar 31, 2022

[NeurIPS2021] Code Release of K-Net: Towards Unified Image Segmentation

K-Net: Towards Unified Image Segmentation Introduction This is an official release of the paper K-Net:Towards Unified Image Segmentation. K-Net will a

423 Jan 2, 2023

PyTorch implementation of Lip to Speech Synthesis with Visual Context Attentional GAN (NeurIPS2021)

Lip to Speech Synthesis with Visual Context Attentional GAN This repository contains the PyTorch implementation of the following paper: Lip to Speech

6 Nov 2, 2022

MAU: A Motion-Aware Unit for Video Prediction and Beyond, NeurIPS2021

MAU (NeurIPS2021) Zheng Chang, Xinfeng Zhang, Shanshe Wang, Siwei Ma, Yan Ye, Xinguang Xiang, Wen GAo. Official PyTorch Code for "MAU: A Motion-Aware

20 Nov 25, 2022

Categorical Depth Distribution Network for Monocular 3D Object Detection

CaDDN CaDDN is a monocular-based 3D object detection method. This repository is based off of [OpenPCDet]. Categorical Depth Distribution Network for M

289 Jan 5, 2023

Comments

ImageNet-30 models

Thank you very much for a very reproducible-oriented repository!

Can you please publish your ImageNet-30 models (ResNet-18 & ResNet-101) so we can run pNML and evaluate OOD detection performance rates on them? I think the easiest way would be to add it to the bash_scripts/download_models.sh file.

Thanks!

opened by ajevnisek 2
Bump pillow from 8.2.0 to 8.3.2
Bumps pillow from 8.2.0 to 8.3.2.

Release notes

Sourced from pillow's releases.

8.3.2

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.2.html

Security

CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

Fix 6-byte OOB read in FliDecode [wiredfool]

Python 3.10 wheels

Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

Fixed regressions

Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

Updates for ImagePalette channel order #5599 [radarhere]

Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

8.3.1

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.1.html

Changes

Catch OSError when checking if fp is sys.stdout #5585 [@radarhere]

Handle removing orientation from alternate types of EXIF data #5584 [@radarhere]

Make Image.array take optional dtype argument #5572 [@t-vi]

8.3.0

https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html

Changes

Use snprintf instead of sprintf #5567 [@radarhere]

Limit TIFF strip size when saving with LibTIFF #5514 [@kmilos]

Allow ICNS save on all operating systems #4526 [@newpanjing]

De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [@gofr]

Do not use background or transparency index for new color #5564 [@radarhere]

Simplified code #5315 [@radarhere]

Replaced xml.etree.ElementTree #5565 [@radarhere]

... (truncated)

Changelog

Sourced from pillow's changelog.

8.3.2 (2021-09-02)

CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

Fix 6-byte OOB read in FliDecode [wiredfool]

Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

Updates for ImagePalette channel order #5599 [radarhere]

Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

8.3.1 (2021-07-06)

Catch OSError when checking if fp is sys.stdout #5585 [radarhere]

Handle removing orientation from alternate types of EXIF data #5584 [radarhere]

Make Image.array take optional dtype argument #5572 [t-vi, radarhere]

8.3.0 (2021-07-01)

Use snprintf instead of sprintf. CVE-2021-34552 #5567 [radarhere]

Limit TIFF strip size when saving with LibTIFF #5514 [kmilos]

Allow ICNS save on all operating systems #4526 [baletu, radarhere, newpanjing, hugovk]

De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [gofr, radarhere]

Replaced xml.etree.ElementTree #5565 [radarhere]

... (truncated)

Commits

8013f13 8.3.2 version bump

23c7ca8 Update CHANGES.rst

8450366 Update release notes

a0afe89 Update test case

9e08eb8 Raise ValueError if color specifier is too long

bd5cf7d FLI tests for Oss-fuzz crash.

94a0cf1 Fix 6-byte OOB read in FliDecode

cece64f Add 8.3.2 (2021-09-02) [CI skip]

e422386 Add release notes for Pillow 8.3.2

08dcbb8 Pillow 8.3.2 supports Python 3.10 [ci skip]

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump pillow from 8.0.1 to 8.1.1
Bumps pillow from 8.0.1 to 8.1.1.

Release notes

Sourced from pillow's releases.

8.1.1

https://pillow.readthedocs.io/en/stable/releasenotes/8.1.1.html

8.1.0

https://pillow.readthedocs.io/en/stable/releasenotes/8.1.0.html

Changes

Fix TIFF OOB Write error #5175 [@radarhere]

Fix for Buffer Read Overrun in PCX Decoding #5174 [@radarhere]

Fix for SGI Decode buffer overrun #5173 [@radarhere]

Fix OOB Read when saving GIF of xsize=1 #5149 [@wiredfool]

Add support for PySide6 #5161 [@hugovk]

Moved QApplication into one test #5167 [@radarhere]

Use disposal settings from previous frame in APNG #5126 [@radarhere]

Revert "skip wheels on 3.10-dev due to wheel#354" #5163 [@radarhere]

Better _binary module use #5156 [@radarhere]

Added exception explaining that repr_png saves to PNG #5139 [@radarhere]

Use previous disposal method in GIF load_end #5125 [@radarhere]

Do not catch a ValueError only to raise another #5090 [@radarhere]

Allow putpalette to accept 1024 integers to include alpha values #5089 [@radarhere]

Fix OOB Read when writing TIFF with custom Metadata #5148 [@wiredfool]

Removed unused variable #5140 [@radarhere]

Fix dereferencing of potential null pointers #5111 [@cgohlke]

Fixed warnings assigning to "unsigned char *" from "char *" #5127 [@radarhere]

Add append_images support for ICO #4568 [@ziplantil]

Fixed comparison warnings #5122 [@radarhere]

Block TIFFTAG_SUBIFD #5120 [@radarhere]

Fix dereferencing potential null pointer #5108 [@cgohlke]

Replaced PyErr_NoMemory with ImagingError_MemoryError #5113 [@radarhere]

Remove duplicate code #5109 [@cgohlke]

Moved warning to end of execution #4965 [@radarhere]

Removed unused fromstring and tostring C methods #5026 [@radarhere]

init() if one of the formats is unrecognised #5037 [@radarhere]

Dependencies

Updated libtiff to 4.2.0 #5153 [@radarhere]

Updated openjpeg to 2.4.0 #5151 [@radarhere]

Updated harfbuzz to 2.7.4 #5138 [@radarhere]

Updated harfbuzz to 2.7.3 #5128 [@radarhere]

Updated libraqm to 0.7.1 #5070 [@radarhere]

Updated libimagequant to 2.13.1 #5065 [@radarhere]

Update FriBiDi to 1.0.10 #5064 [@nulano]

Updated libraqm to 0.7.1 #5063 [@radarhere]

Updated libjpeg-turbo to 2.0.6 #5044 [@radarhere]

Deprecations

... (truncated)

Changelog

Sourced from pillow's changelog.

8.1.1 (2021-03-01)

Use more specific regex chars to prevent ReDoS. CVE-2021-25292 [hugovk]

Fix OOB Read in TiffDecode.c, and check the tile validity before reading. CVE-2021-25291 [wiredfool]

Fix negative size read in TiffDecode.c. CVE-2021-25290 [wiredfool]

Fix OOB read in SgiRleDecode.c. CVE-2021-25293 [wiredfool]

Incorrect error code checking in TiffDecode.c. CVE-2021-25289 [wiredfool]

PyModule_AddObject fix for Python 3.10 #5194 [radarhere]

8.1.0 (2021-01-02)

Fix TIFF OOB Write error. CVE-2020-35654 #5175 [wiredfool]

Fix for Read Overflow in PCX Decoding. CVE-2020-35653 #5174 [wiredfool, radarhere]

Fix for SGI Decode buffer overrun. CVE-2020-35655 #5173 [wiredfool, radarhere]

Fix OOB Read when saving GIF of xsize=1 #5149 [wiredfool]

Makefile updates #5159 [wiredfool, radarhere]

Add support for PySide6 #5161 [hugovk]

Use disposal settings from previous frame in APNG #5126 [radarhere]

Added exception explaining that repr_png saves to PNG #5139 [radarhere]

Use previous disposal method in GIF load_end #5125 [radarhere]

... (truncated)

Commits

741d874 8.1.1 version bump

179cd1c Added 8.1.1 release notes to index

7d29665 Update CHANGES.rst [ci skip]

d25036f Credits

973a4c3 Release notes for 8.1.1

521dab9 Use more specific regex chars to prevent ReDoS

8b8076b Fix for CVE-2021-25291

e25be1e Fix negative size read in TiffDecode.c

f891baa Fix OOB read in SgiRleDecode.c

cbfdde7 Incorrect error code checking in TiffDecode.c

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump numpy from 1.19.1 to 1.22.0
Bumps numpy from 1.19.1 to 1.22.0.

Release notes

Sourced from numpy's releases.

v1.22.0

NumPy 1.22.0 Release Notes

NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.

A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.

NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.

New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.

A new configurable allocator for use by downstream projects.

These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

Expired deprecations

Deprecated numeric style dtype strings have been removed

Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

(gh-19539)

Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

(gh-19615)

... (truncated)

Commits

4adc87d Merge pull request #20685 from charris/prepare-for-1.22.0-release

fd66547 REL: Prepare for the NumPy 1.22.0 release.

125304b wip

c283859 Merge pull request #20682 from charris/backport-20416

5399c03 Merge pull request #20681 from charris/backport-20954

f9c45f8 Merge pull request #20680 from charris/backport-20663

794b36f Update armccompiler.py

d93b14e Update test_public_api.py

7662c07 Update init.py

311ab52 Update armccompiler.py

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0