PyTorch code for the ICCV'21 paper: "Always Be Dreaming: A New Approach for Class-Incremental Learning"

Last update: Dec 21, 2022

Related tags

Deep Learning AlwaysBeDreaming-DFCIL

Overview

Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

PyTorch code for the ICCV 2021 paper:
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning
James Smith, Yen-Chang Hsu, Jonathan Balloch, Yilin Shen, Hongxia Jin, Zsolt Kira
International Conference on Computer Vision (ICCV), 2021
[arXiv] [pdf] [project]

Abstract

Modern computer vision applications suffer from catastrophic forgetting when incrementally learning new concepts over time. The most successful approaches to alleviate this forgetting require extensive replay of previously seen data, which is problematic when memory constraints or data legality concerns exist. In this work, we consider the high-impact problem of Data-Free Class-Incremental Learning (DFCIL), where an incremental learning agent must learn new concepts over time without storing generators or training data from past tasks. One approach for DFCIL is to replay synthetic images produced by inverting a frozen copy of the learner's classification model, but we show this approach fails for common class-incremental benchmarks when using standard distillation strategies. We diagnose the cause of this failure and propose a novel incremental distillation strategy for DFCIL, contributing a modified cross-entropy training and importance-weighted feature distillation, and show that our method results in up to a 25.1% increase in final task accuracy (absolute difference) compared to SOTA DFCIL methods for common class-incremental benchmarks. Our method even outperforms several standard replay based methods which store a coreset of images.

Installation

Prerequisites

python == 3.6
torch == 1.0.1
torchvision >= 0.2.1

Setup

Install anaconda: https://www.anaconda.com/distribution/
set up conda environmet w/ python 3.6, ex: conda create --name py36 python=3.6
conda activate py36
sh install_requirements.sh

Datasets

Download/Extract the following datasets to the dataset folder under the project root directory.

For CIFAR-10 and CIFAR-100, download the python version dataset here.

Training

All commands should be run under the project root directory.

sh experiments/cifar100-fivetask.sh # tables 1,2
sh experiments/cifar100-tentask.sh # tables 1,2
sh experiments/cifar100-twentytask.sh # tables 1,2

Results

Results are generated for various task sizes. See the main text for full details. Numbers represent final accuracy in three runs (higher the better).

CIFAR-100 (no coreset)

tasks	5	10	20
UB	69.9 ± 0.2	69.9 ± 0.2	69.9 ± 0.2
Base	16.4 ± 0.4	8.8 ± 0.1	4.4 ± 0.3
LwF	17.0 ± 0.1	9.2 ± 0.0	4.7 ± 0.1
LwF.MC	32.5 ± 1.0	17.1 ± 0.1	7.7 ± 0.5
DGR	14.4 ± 0.4	8.1 ± 0.1	4.1 ± 0.3
DeepInversion	18.8 ± 0.3	10.9 ± 0.6	5.7 ± 0.3
Ours	43.9 ± 0.9	33.7 ± 1.2	20.0 ± 1.4

CIFAR-100 (with 2000 image coreset)

tasks	5	10	20
UB	69.9 ± 0.2	69.9 ± 0.2	69.9 ± 0.2
Naive Rehearsal	34.0 ± 0.2	24.0 ± 1.0	14.9 ± 0.7
LwF	39.4 ± 0.3	27.4 ± 0.8	16.6 ± 0.4
E2E	47.4 ± 0.8	38.4 ± 1.3	32.7 ± 1.9
BiC	53.7 ± 0.4	45.9 ± 1.8	37.5 ± 3.2
Ours (no coreset)	43.9 ± 0.9	33.7 ± 1.2	20.0 ± 1.4

Acknowledgement

This work is supported by Samsung Research America.

Citation

If you found our work useful for your research, please cite our work:

@article{smith2021always,
  author    = {Smith, James and Hsu, Yen-Chang and Balloch, Jonathan and Shen, Yilin and Jin, Hongxia and Kira, Zsolt},
  title     = {Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  month     = {October},
  year      = {2021},
  pages     = {9374-9384}
}

Comments

What means about 'importance-weighted' in paper？

Nice work, but i have some little question.

What means about 'importance-weighted' in paper?

in code, ### def data_weighting() in default.py, i find self.dw_k is [1,1,1,1,........1]

### class AlwaysBeDreaming(DeepInversionGenBN) in datafree.py the loss_kd = self.kd_criterion(logits_KD, logits_KD_past).sum(dim=1) * dw_KD dw_KD is also [1,1,1,......1]

it's not useful,and i don't understead where means 'importance-weighted' or 'data weighting'?

opened by miss-rain 8
Resnet backbone in ImageNet experiments

Dear James, Thanks for sharing the code of your method! A quick question regarding the ResNet architecture used in experiments/imnet-fivetask.sh and experiments/tinyimnet-twentytask.sh You have "MODELNAME=resnet32" in both scripts. Shouldn't this be resnet18 as stated in the paper? Thanks!

opened by ibadiu 1
Question about the implementation of LwF

Nice work!

I have a question about your implementation of LwF, it seems that there is a big gap between the data of LwF reported in your paper and the previous works (iCaRL and End-to-End Incremental Learning).

I also implement the LwF.MC introduced by iCaRL, and my results on CIFAR100 consistent with the data reported by iCaRL, there are also some other implementations, such as PODNet, it seems that PODNet uses the BCELoss, while I use the CrossEntropyLoss instead.

I read your code, the problem may lie in the loss in LwF.MC.

opened by gqk 1
The struture of generator of ImageNet benchmark

Nice work! But i have some little question. (1)Could you provide the structure details of generators on ImageNet and TinyImageNet benchmarks? i.e., F_\phi in Eq(7). (2)Could you publish the code of ImageNet and TinyImageNet benchmarks?

opened by XiaorongLi-95 1
Why validating model without taking "with torch.no_grad()"?

Thanks for your valuable project.

Below I found that you take with torch.no_grad(): on the dataset (i.e., input and target in lines 210-211) rather than on the model in line 213. As shown in Evaluating pytorch models: with torch.no_grad vs model.eval() and ‘model.eval()’ vs ‘with torch.no_grad()’, it will take too much memory. But I am not sure about taking no with torch.no_grad block on line 213 whether influences the model's outputs.

https://github.com/GT-RIPL/AlwaysBeDreaming-DFCIL/blob/50d9c2e9857ef217917234e7c7a7ed493f55aed2/learners/default.py#L206-L226

opened by HLBayes 0
CVE-2007-4559 Patch

Patching CVE-2007-4559

Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

opened by TrellixVulnTeam 0

PyTorch code for the ICCV'21 paper: "Always Be Dreaming: A New Approach for Class-Incremental Learning"

Related tags

Overview

Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

Abstract

Installation

Prerequisites

Setup

Datasets

Training

Results

CIFAR-100 (no coreset)

CIFAR-100 (with 2000 image coreset)

Acknowledgement

Citation

Comments

What means about 'importance-weighted' in paper？

Resnet backbone in ImageNet experiments

Question about the implementation of LwF

The struture of generator of ImageNet benchmark

Why validating model without taking "with torch.no_grad()"?

CVE-2007-4559 Patch

Patching CVE-2007-4559

Owner

[ICCV21] Self-Calibrating Neural Radiance Fields

This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

The LaTeX and Python code for generating the paper, experiments' results and visualizations reported in each paper is available (whenever possible) in the paper's directory

Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

A PyTorch implementation of the paper Mixup: Beyond Empirical Risk Minimization in PyTorch

HashNeRF-pytorch - Pure PyTorch Implementation of NVIDIA paper on Instant Training of Neural Graphics primitives

Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

PyTorch code for ICLR 2021 paper Unbiased Teacher for Semi-Supervised Object Detection

PyTorch code for the paper "Curriculum Graph Co-Teaching for Multi-target Domain Adaptation" (CVPR2021)

PyTorch code for the paper "FIERY: Future Instance Segmentation in Bird's-Eye view from Surround Monocular Cameras"

Pytorch reimplement of the paper "A Novel Cascade Binary Tagging Framework for Relational Triple Extraction" ACL2020. The original code is written in keras.

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

PyTorch code of my ICDAR 2021 paper Vision Transformer for Fast and Efficient Scene Text Recognition (ViTSTR)

Official PyTorch code for CVPR 2020 paper "Deep Active Learning for Biased Datasets via Fisher Kernel Self-Supervision"

PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation

This is the pytorch code for the paper Curious Representation Learning for Embodied Intelligence.

PyTorch code for our paper "Image Super-Resolution with Non-Local Sparse Attention" (CVPR2021).

Pytorch code for ICRA'21 paper: "Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation"