Corruption Invariant Learning for Re-identification

Related tags

Deep Learning corruption robustness reidentification
Overview

Corruption Invariant Learning for Re-identification

The official repository for Benchmarks for Corruption Invariant Person Re-identification (NeurIPS 2021 Track on Datasets and Benchmarks), with exhaustive study on corruption invariant learning in single- and cross-modality ReID datasets, including Market-1501-C, CUHK03-C, MSMT17-C, SYSU-MM01-C, RegDB-C.

PWC PWC PWC PWC PWC

Maintenance Plan

The benchmark will be maintained by the authors. We will get constant lectures about the new proposed ReID models and evaluate them under the CIL benchmark settings in time. Besides, we gladly take feedback to the CIL benchmark and welcome any contributions in terms of the new ReID models and corresponding evaluations. Please feel free to contact us, [email protected] .

TODO:

  • other datasets configurations
  • get started tutorial
  • more detailed statistical evaluations
  • checkpoints of the baseline models
  • cross-modality preson Re-ID dataset, CUHK-PEDES
  • other ReID datasets, like VehicleID, VeRi-776, etc.

(Note: codebase from TransReID)

Quick Start

1. Install dependencies

  • python=3.7.0
  • pytorch=1.6.0
  • torchvision=0.7.0
  • timm=0.4.9
  • albumentations=0.5.2
  • imagecorruptions=1.1.2
  • h5py=2.10.0
  • cython=0.29.24
  • yacs=0.1.6

2. Prepare dataset

Download the datasets, Market-1501, CUHK03, MSMT17. Set the root path of the dataset in congigs/Market/resnet_base.yml, DATASETS: ROOT_DIR: ('root'), or set it in scripts/train_market.sh, DATASETS.ROOT_DIR "('root')".

3. Train

Train a CIL model on Market-1501,

sh ./scripts/train_market.sh

4. Test

Test the CIL model on Market-1501,

sh ./scripts/eval_market.sh

Evaluating Corruption Robustness On-the-fly

Corruption Transform

The main code of corruption transform. (See contextual code in ./datasets/make_dataloader.py, line 59)

from imagecorruptions.corruptions import *

corruption_function = [gaussian_noise, shot_noise, impulse_noise, defocus_blur,
    glass_blur, motion_blur, zoom_blur, snow, frost, fog, brightness, contrast,
    elastic_transform, pixelate, jpeg_compression, speckle_noise,
    gaussian_blur, spatter, saturate, rain]
    
class corruption_transform(object):
    def __init__(self, level=0, type='all'):
        self.level = level
        self.type = type

    def __call__(self, img):
        if self.level > 0 and self.level < 6:
            level_idx = self.level
        else:
            level_idx = random.choice(range(1, 6))
        if self.type == 'all':
            corrupt_func = random.choice(corruption_function)
        else:
            func_name_list = [f.__name__ for f in corruption_function]
            corrupt_idx = func_name_list.index(self.type)
            corrupt_func = corruption_function[corrupt_idx]
        c_img = corrupt_func(img.copy(), severity=level_idx)
        img = Image.fromarray(np.uint8(c_img))
        return img

Evaluating corruption robustness can be realized on-the-fly by modifing the transform function uesed in test dataloader. (See details in ./datasets/make_dataloader.py, Line 266)

val_with_corruption_transforms = T.Compose([
    corruption_transform(0),
    T.Resize(cfg.INPUT.SIZE_TEST),
    T.ToTensor(),])

Rain details

We introduce a rain corruption type, which is a common type of weather condition, but it is missed by the original corruption benchmark. (See details in ./datasets/make_dataloader.py, Line 27)

def rain(image, severity=1):
    if severity == 1:
        type = 'drizzle'
    elif severity == 2 or severity == 3:
        type = 'heavy'
    elif severity == 4 or severity == 5:
        type = 'torrential'
    blur_value = 2 + severity
    bright_value = -(0.05 + 0.05 * severity)
    rain = abm.Compose([
        abm.augmentations.transforms.RandomRain(rain_type=type, 
        blur_value=blur_value, brightness_coefficient=1, always_apply=True),
        abm.augmentations.transforms.RandomBrightness(limit=[bright_value, 
        bright_value], always_apply=True)])
    width, height = image.size
    if height <= 60:
        scale_factor = 65.0 / height
        new_size = (int(width * scale_factor), 65)
        image = image.resize(new_size)
    return rain(image=np.array(image))['image']

Baselines

  • Single-modality datasets
                                                                                   
Dataset Method Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
Market-1501 BoT 59.30 85.06 93.38 0.20 8.42 27.05
AGW 64.03 86.51 94.00 0.35 12.13 31.90
SBS 60.03 88.33 95.90 0.29 11.54 34.13
CIL (ours) 57.90 84.04 93.38 1.76 (0.13) 28.03 (0.45) 55.57 (0.63)
MSMT17 BoT 9.91 48.34 73.53 0.07 5.28 20.20
AGW 12.38 51.84 75.21 0.08 6.53 22.77
SBS 10.26 56.62 82.02 0.05 7.89 28.77
CIL (ours) 12.45 52.40 76.10 0.32 (0.03) 15.33 (0.20) 39.79 (0.45)
CUHK03  AGW   49.97   62.25   64.64   0.46   3.45  5.90 
 CIL (ours)   53.87   65.16   67.29   4.25 (0.39)   16.33 (0.76)   22.96 (1.04) 
  • Cross-modality datasets

Note: For RegDB dataset, Mode A and Mode B represent visible-to-thermal and thermal-to-visible experimental settings, respectively. And for SYSU-MM01 dataset, Mode A and Mode B represent all search and indoor search respectively. Note that we only corrupt RGB (visible) images in the corruption evaluation.

                                                                                                                                                                                                                                                                     
Dataset Method Mode A Mode B
Clean Eval. Corruption Eval. Clean Eval. Corruption Eval.
mINP mAP R-1 mINP mAP R-1 mINP mAP R-1 mINP mAP R-1
SYSU-MM01  AGW   36.17   47.65   47.50   14.73   29.99   34.42   59.74   62.97   54.17   35.39   40.98   33.80 
 CIL (ours)   38.15   47.64   45.51   22.48 (1.65)   35.92 (1.22)   36.95 (0.67)   57.41   60.45   50.98   43.11 (4.19)   48.65 (4.57)   40.73 (5.55) 
RegDB  AGW   54.10   68.82   75.78   32.88   43.09   45.44   52.40   68.15   75.29   6.00   41.37   67.54 
 CIL (ours)   55.68   69.75   74.96   38.66 (0.01)   49.76 (0.03)   52.25 (0.03)   55.50   69.21   74.95   11.94 (0.12)   47.90 (0.01)   67.17 (0.06)

Recent Advance in Person Re-ID

Leaderboard

Market1501-C

(Note: ranked by mAP on corrupted test set)

Method Reference Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
TransReID Shuting He et al. (2021) 69.29 88.93 95.07 1.98 27.38 53.19
CaceNet Fufu Yu et al. (2020) 70.47 89.82 95.40 0.67 18.24 42.92
LightMBN Fabian Herzog et al. (2021) 73.29 91.54 96.53 0.50 14.84 38.68
PLR-OS Ben Xie et al. (2020) 66.42 88.93 95.19 0.48 14.23 37.56
RRID Hyunjong Park et al. (2019) 67.14 88.43 95.19 0.46 13.45 36.57
Pyramid Feng Zheng et al. (2018) 61.61 87.50 94.86 0.36 12.75 35.72
PCB Yifan Sun et al.(2017) 41.97 82.19 94.15 0.41 12.72 34.93
BDB Zuozhuo Dai et al. (2018) 61.78 85.47 94.63 0.32 10.95 33.79
Aligned++ Hao Luo et al. (2019) 47.31 79.10 91.83 0.32 10.95 31.00
AGW Mang Ye et al. (2020) 65.40 88.10 95.00 0.30 10.80 33.40
MHN Binghui Chen et al. (2019) 55.27 85.33 94.50 0.38 10.69 33.29
LUPerson Dengpan Fu et al. (2020) 68.71 90.32 96.32 0.29 10.37 32.22
OS-Net Kaiyang Zhou et al. (2019) 56.78 85.67 94.69 0.23 10.37 30.94
VPM Yifan Sun et al. (2019) 50.09 81.43 93.79 0.31 10.15 31.17
DG-Net Zhedong Zheng et al. (2019) 61.60 86.09 94.77 0.35 9.96 31.75
ABD-Net Tianlong Chen et al. (2019) 64.72 87.94 94.98 0.26 9.81 29.65
MGN Guanshuo Wang et al.(2018) 60.86 86.51 93.88 0.29 9.72 29.56
F-LGPR Yunpeng Gong et al. (2021) 65.48 88.22 95.37 0.23 9.08 29.35
TDB Rodolfo Quispe et al. (2020) 56.41 85.77 94.30 0.20 8.90 28.56
LGPR Yunpeng Gong et al. (2021) 58.71 86.09 94.51 0.24 8.26 27.72
BoT Hao Luo et al. (2019) 51.00 83.90 94.30 0.10 6.60 26.20

CUHK03-C (detected)

(Note: ranked by mAP on corrupted test set)

Method Reference Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
CaceNet Fufu Yu et al. (2020) 65.22 75.13 77.64 2.09 10.62 17.04
Pyramid Feng Zheng et al. (2018) 61.41 73.14 79.54 1.10 8.03 10.42
RRID Hyunjong Park et al. (2019) 55.81 67.63 74.99 1.00 7.30 9.66
PLR-OS Ben Xie et al. (2020) 62.72 74.67 78.14 0.89 6.49 10.99
Aligned++ Hao Luo et al. (2019) 47.32 59.76 62.07 0.56 4.87 7.99
MGN Guanshuo Wang et al.(2018) 51.18 62.73 69.14 0.46 4.20 5.44
MHN Binghui Chen et al. (2019) 56.52 66.77 72.21 0.46 3.97 8.27

MSMT17-C (Version 2)

(Note: ranked by mAP on corrupted test set)

Method Reference Clean Eval. Corruption Eval.
mINP mAP Rank-1 mINP mAP Rank-1
OS-Net Kaiyang Zhou et al. (2019) 4.05 40.05 71.86 0.08 7.86 28.51
AGW Mang Ye et al. (2020) 12.38 51.84 75.21 0.08 6.53 22.77
BoT Hao Luo et al. (2019) 9.91 48.34 73.53 0.07 5.28 20.20

Citation

Kindly include a reference to this paper in your publications if it helps your research:

@misc{chen2021benchmarks,
    title={Benchmarks for Corruption Invariant Person Re-identification},
    author={Minghui Chen and Zhiqiang Wang and Feng Zheng},
    year={2021},
    eprint={2111.00880},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
Comments
  • Why did I get so much higher numbers than you did in your paper?

    Why did I get so much higher numbers than you did in your paper?

    Thanks for your works. As you set up in your code, I used TransReID to reproduce on the dataset Market1501-C, but I got an incredible result without any modity. mINP=6.3, mAP=40.45, Rank-1=65.05 (Corrupted eval). Is there something I'm not setting right？

    opened by ChuxiaYang 2
  • when Test,TypeError

    when Test,TypeError

    Hi, thanks for your work! It's something I can learn from, but I test my model at the end of the training, is show up Traceback (most recent call last): File "F:/workspace/CIL-ReID-main/test.py", line 72, in mINP, mAP, rank1, rank5, rank10 = do_inference( File "F:\workspace\CIL-ReID-main\processor\processor.py", line 208, in do_inference logger.info("mINP: {:.2%}".format(mINP)) TypeError: unsupported format string passed to list.format

    I queried the use of format in Python, but to no avail. Change the error position to:

    logger.info("mINP: {:.2%}".format(str(mINP))) logger.info("mAP: {:.2%}".format(str(mAP))),

    there will be a new series of mistakes. I hope you can help me solve the problem! Thank you! I sincerely!

    opened by syh-hue 0
  • cuhk03dataset

    cuhk03dataset

    Excuse me, the structure of cuhk03 dataset does not have the three folders bounding_box_train, query and bounding_box_test, how should I run your code on this dataset?

    opened by LY-cmd-xtt 0
  • about Cython

    about Cython

    hello! I got an error before training:

    (reid) liyuke@song12306:~/CIL-ReID-main$ python train.py
Traceback (most recent call last):
  File "train.py", line 7, in <module>
    from processor import do_train
  File "/home/liyuke/CIL-ReID-main/processor/__init__.py", line 1, in <module>
    from .processor import do_train, do_inference
  File "/home/liyuke/CIL-ReID-main/processor/processor.py", line 7, in <module>
    from utils.metrics import R1_mAP_eval
  File "/home/liyuke/CIL-ReID-main/utils/metrics.py", line 5, in <module>
    from .rank_cylib.rank_cy import evaluate_cy
ModuleNotFoundError: No module named 'utils.rank_cylib.rank_cy'

    instruct:

    python train.py

    and my env list:

    Package                       Version
----------------------------- -----------
absl-py                       1.0.0
albumentations                1.1.0
anaconda-client               1.9.0
astunparse                    1.6.3
attrs                         21.4.0
backcall                      0.2.0
backports.functools-lru-cache 1.6.4
brotlipy                      0.7.0
cachetools                    4.2.4
certifi                       2021.10.8
cffi                          1.15.0
charset-normalizer            2.0.4
chex                          0.1.0
cloudpickle                   2.0.0
clu                           0.0.6
clyent                        1.2.2
contextlib2                   21.6.0
cryptography                  36.0.0
cycler                        0.11.0
Cython                        0.29.28
debugpy                       1.5.1
decorator                     5.1.0
dill                          0.3.4
dm-tree                       0.1.6
einops                        0.3.2
entrypoints                   0.4
faiss-gpu                     1.7.1.post3
flatbuffers                   2.0
flax                          0.3.6
fonttools                     4.28.5
gast                          0.4.0
google-auth                   2.3.3
google-auth-oauthlib          0.4.6
google-pasta                  0.2.0
googleapis-common-protos      1.54.0
grpcio                        1.43.0
h5py                          3.6.0
idna                          3.3
imagecorruptions              1.1.2
imageio                       2.16.2
importlib-metadata            4.11.3
importlib-resources           5.4.0
ipdb                          0.13.9
ipykernel                     6.11.0
ipython                       7.30.0
ipython-genutils              0.2.0
jax                           0.2.26
jaxlib                        0.1.75
jedi                          0.18.1
joblib                        1.1.0
jsonschema                    3.2.0
jupyter-client                7.0.6
jupyter-core                  4.9.2
keras                         2.7.0
Keras-Preprocessing           1.1.2
kiwisolver                    1.3.2
libclang                      12.0.0
lmdb                          1.3.0
matplotlib                    3.5.1
matplotlib-inline             0.1.3
mkl-fft                       1.3.1
mkl-random                    1.2.2
mkl-service                   2.4.0
ml-collections                0.1.0
msgpack                       1.0.3
nbformat                      5.1.3
nest-asyncio                  1.5.4
networkx                      2.8
numpy                         1.22.1
olefile                       0.46
opencv-python                 4.5.5.62
opencv-python-headless        4.5.5.64
opt-einsum                    3.3.0
optax                         0.1.0
packaging                     21.3
parso                         0.8.2
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        8.4.0
pip                           21.2.4
promise                       2.3
prompt-toolkit                3.0.22
protobuf                      3.19.3
psutil                        5.8.0
ptyprocess                    0.7.0
pycparser                     2.21
Pygments                      2.10.0
pyOpenSSL                     22.0.0
pyparsing                     3.0.6
pyrsistent                    0.18.0
PySocks                       1.7.1
python-dateutil               2.8.2
pytorch-ignite                0.1.2
pytz                          2021.3
PyWavelets                    1.3.0
PyYAML                        5.4.1
pyzmq                         19.0.2
qudida                        0.0.4
requests                      2.27.1
requests-oauthlib             1.3.0
rsa                           4.8
scikit-image                  0.19.2
scikit-learn                  1.0.2
scipy                         1.7.3
setuptools                    58.0.4
six                           1.16.0
sklearn                       0.0
tensorboard                   2.7.0
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorflow                    2.7.0
tensorflow-cpu                2.7.0
tensorflow-datasets           4.4.0
tensorflow-estimator          2.7.0
tensorflow-io-gcs-filesystem  0.23.1
tensorflow-metadata           1.5.0
tensorflow-probability        0.15.0
termcolor                     1.1.0
threadpoolctl                 3.0.0
tifffile                      2022.4.8
timm                          0.4.12
toml                          0.10.2
toolz                         0.11.2
torch                         1.10.0
torchaudio                    0.10.0
torchvision                   0.11.1
tornado                       6.1
tqdm                          4.62.3
traitlets                     5.1.1
typing_extensions             4.0.1
urllib3                       1.26.8
wcwidth                       0.2.5
Werkzeug                      2.0.2
wheel                         0.37.0
wrapt                         1.13.3
yacs                          0.1.6
zipp                          3.7.0
    opened by liyuke65535 1
  • How to use the transformer model on this Robust ReID task?

    How to use the transformer model on this Robust ReID task?

    The codebase is a very nice work. I just find the script file of using the resnet to train and would like to know that if you have the plan to release the script of transformer model. Look forward for your reply.

    Best wishes.

    opened by zkcys001 1
Owner
Minghui Chen
Minghui Chen
GitHub
Learning Domain Invariant Representations in Goal-conditioned Block MDPs

Learning Domain Invariant Representations in Goal-conditioned Block MDPs Beining Han, Chongyi Zheng, Harris Chan, Keiran Paster, Michael R. Zhang, Jim

Chongyi Zheng 3 Apr 12, 2022
Code for NeurIPS 2021 paper: Invariant Causal Imitation Learning for Generalizable Policies

Invariant Causal Imitation Learning for Generalizable Policies Ioana Bica, Daniel Jarrett, Mihaela van der Schaar Neural Information Processing System

Ioana Bica 17 Dec 1, 2022
The Python ensemble sampling toolkit for affine-invariant MCMC

emcee The Python ensemble sampling toolkit for affine-invariant MCMC emcee is a stable, well tested Python implementation of the affine-invariant ense

Dan Foreman-Mackey 1.3k Dec 31, 2022
《Truly shift-invariant convolutional neural networks》(2021)

Truly shift-invariant convolutional neural networks [Paper] Authors: Anadi Chaman and Ivan Dokmanić Convolutional neural networks were always assumed

Anadi Chaman 46 Dec 19, 2022
Expressive Power of Invariant and Equivaraint Graph Neural Networks (ICLR 2021)

Expressive Power of Invariant and Equivaraint Graph Neural Networks In this repository, we show how to use powerful GNN (2-FGNN) to solve a graph alig

Marc Lelarge 36 Dec 12, 2022
Systematic generalisation with group invariant predictions

Requirements are Python 3, TensorFlow v1.14, Numpy, Scipy, Scikit-Learn, Matplotlib, Pillow, Scikit-Image, h5py, tqdm. Experiments were run on V100 GPUs (16 and 32GB).

Faruk Ahmed 30 Dec 1, 2022
PERIN is Permutation-Invariant Semantic Parser developed for MRP 2020

PERIN: Permutation-invariant Semantic Parsing David Samuel & Milan Straka Charles University Faculty of Mathematics and Physics Institute of Formal an

ÚFAL 40 Jan 4, 2023
Implementation of Invariant Point Attention, used for coordinate refinement in the structure module of Alphafold2, as a standalone Pytorch module

Invariant Point Attention - Pytorch Implementation of Invariant Point Attention as a standalone module, which was used in the structure module of Alph

Phil Wang 113 Jan 5, 2023
Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer"

SCGAN Implementation of CVPR 2021 paper "Spatially-invariant Style-codes Controlled Makeup Transfer" Prepare The pre-trained model is avaiable at http

null 118 Dec 12, 2022
A PyTorch Implementation of Single Shot Scale-invariant Face Detector.

S³FD: Single Shot Scale-invariant Face Detector A PyTorch Implementation of Single Shot Scale-invariant Face Detector. Eval python wider_eval_pytorch.

carwin 235 Jan 7, 2023
PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features

PRIN/SPRIN: On Extracting Point-wise Rotation Invariant Features Overview This repository is the Pytorch implementation of PRIN/SPRIN: On Extracting P

Yang You 17 Mar 2, 2022
This is an example implementation of the paper "Cross Domain Robot Imitation with Invariant Representation".

IR-GAIL This is an example implementation of the paper "Cross Domain Robot Imitation with Invariant Representation". Dependency The experiments are de

Zhao-Heng Yin 1 Jul 14, 2022
An SE(3)-invariant autoencoder for generating the periodic structure of materials

Crystal Diffusion Variational AutoEncoder This software implementes Crystal Diffusion Variational AutoEncoder (CDVAE), which generates the periodic st

Tian Xie 94 Dec 10, 2022
DIR-GNN - Discovering Invariant Rationales for Graph Neural Networks

DIR-GNN "Discovering Invariant Rationales for Graph Neural Networks" (ICLR 2022)

Ying-Xin (Shirley) Wu 70 Nov 13, 2022
The code for our paper submitted to RAL/IROS 2022: OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for LiDAR-Based Place Recognition.

OverlapTransformer The code for our paper submitted to RAL/IROS 2022: OverlapTransformer: An Efficient and Rotation-Invariant Transformer Network for

HAOMO.AI 136 Jan 3, 2023
[TIP2020] Adaptive Graph Representation Learning for Video Person Re-identification

Introduction This is the PyTorch implementation for Adaptive Graph Representation Learning for Video Person Re-identification. Get started git clone h

WuYiming 41 Dec 12, 2022
IAUnet: Global Context-Aware Feature Learning for Person Re-Identification

IAUnet This repository contains the code for the paper: IAUnet: Global Context-Aware Feature Learning for Person Re-Identification Ruibing Hou, Bingpe

null 30 Jul 14, 2022
Implementation of "Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification"

hypergraph_reid Implementation of "Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification" If you find this help your research,

null 62 Dec 21, 2022
Code reproduce for paper "Vehicle Re-identification with Viewpoint-aware Metric Learning"

VANET Code reproduce for paper "Vehicle Re-identification with Viewpoint-aware Metric Learning" Introduction This is the implementation of article VAN

EMDATA-AILAB 23 Dec 26, 2022