This is a pytorch implementation of the NeurIPS paper GAN Memory with No Forgetting.

Related tags

Deep Learning GANmemory_LifelongLearning
Overview

GAN Memory for Lifelong learning

This is a pytorch implementation of the NeurIPS paper GAN Memory with No Forgetting.

Please consider citing our paper if you refer to this code in your research.

@article{cong2020gan,
  title={GAN Memory with No Forgetting},
  author={Cong, Yulai and Zhao, Miaoyun and Li, Jianqiao and Wang, Sijia and Carin, Lawrence},
  journal={Advances in Neural Information Processing Systems},
  volume={33},
  year={2020}
}

Requirement

python=3.7.3
pytorch=1.2.0

Notes

The source model is based on the GP-GAN.

GANMemory_Flowers.py is the implementation of the model in Figure1(a).

classConditionGANMemory.py is the class-conditional generalization of GAN memory, which is used as pseudo rehearsal for a lifelong classification as shown in Section 5.2.

Lifelong_classification.py is the code for the lifelong classification part as shown in Section 5.2.

Usage

First, download the pretrained GP-GAN model by running download_pretrainedGAN.py. Note please change the path therein.

Second, download the training data to the folder ./data/. For example, download the Flowers dataset from: https://www.robots.ox.ac.uk/~vgg/data/flowers/102/ to the folder ./data/102flowers/.

Dataset preparation

data
├──102flowers
           ├──all8189images
├── CelebA
...

Finally, run GANMemory_Flowers.py.

The FID scores of our method shown in Figure 1(b) are summerized in the following table.

Dataset 5K 10K 15K 20K 25K 30K 35K 40K 45K 50K 55K 60K
Flowers 29.26 23.25 19.73 17.98 17.04 16.10 15.93 15.38 15.33 14.96 15.19 14.75
Cathedrals 19.78 18.32 17.10 16.47 16.15 16.33 16.08 15.94 15.78 15.60 15.64 15.67
Cats 38.56 25.74 23.14 21.15 20.80 20.89 19.73 19.88 18.69 18.57 17.57 18.18

For lifelong classification

  1. run classConditionGANMemory.py for each task until the whole sequeence of tasks are remembered and save the generators;

  2. run Lifelong_classification.py to get the classification results.

  3. run Compression_low_rank_six_butterfly.py to get the compression results.

Note, for the sake of simplicity, we devide the pseudo rehearsal based lifelong classification processes into above two stages, one can of course find a way to merge these two stages to form a learning process along task sequence.

Acknowledgement

Our code is based on GAN_stability: https://github.com/LMescheder/GAN_stability from the paper Which Training Methods for GANs do actually Converge?.

Comments
  • Some functions used are missing

    Some functions used are missing

    Hi there, Thanks for the code and paper! I tried to run the code, but ran into the following issue:

    • Trainer does not have argument D_fix_layer in gan_training/train.py (This stops me from running any experiments): Traceback (most recent call last): File "GANMemory_Flowers.py", line 224, in D_fix_layer=config['discriminator']['layers'] TypeError: init() got an unexpected keyword argument 'D_fix_layer'

    • Missing calculate_fid_given_real_ms function used by eval.py

    It would be really helpful if you could answer the above questions.

    Thanks in advance.

    opened by qinenergy 8
  • the pretrained model

    the pretrained model

    Thanks for your codes uploaded. The urls of pretrained models are unavailable maybe the amazonaws server is on due. Is there any other way to download the pretrained models? Best regards.

    opened by leatherking 2
  • Model with further compression

    Model with further compression

    Hi, Thanks for this excellent paper. Could you upload the model with further compression? I think compression is one of the highlights of the article.

    opened by ZiruiYan 2
  • How to enable DistributedDataParallel?

    How to enable DistributedDataParallel?

    Hi Miaoyun,

    Thank you for your great work. I realize that it is time-consuming in training a single experiment. Therefore, I wish to ask you a favor that how to enable the DDP in pytorch of your method?

    thanks. I tried to enable DDP based on your implementation, but the results seem quite strange and less standard. thank you for your consideration.

    opened by TropicLurker 0
  • ImageNet categories chosen

    ImageNet categories chosen

    In the paper, for lifelong classification on ImageNet, there are six animals chosen and each contains 6 categories. What are the specific IDs for the 36 categories that have been used? Thanks!

    opened by eric11220 1
Owner
Miaoyun Zhao
Miaoyun Zhao
GitHub
Official code of CVPR 2021's PLOP: Learning without Forgetting for Continual Semantic Segmentation

PLOP: Learning without Forgetting for Continual Semantic Segmentation This repository contains all of our code. It is a modified version of Cermelli e

Arthur Douillard 116 Dec 14, 2022
Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Memory Efficient Attention Pytorch Implementation of a memory efficient multi-head attention as proposed in the paper, Self-attention Does Not Need O(

Phil Wang 180 Jan 5, 2023
PyTorch Code of "Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics"

Memory In Memory Networks It is based on the paper Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spati

Yang Li 12 May 30, 2022
Segcache: a memory-efficient and scalable in-memory key-value cache for small objects

Segcache: a memory-efficient and scalable in-memory key-value cache for small objects This repo contains the code of Segcache described in the followi

TheSys Group @ CMU CS 78 Jan 7, 2023
Episodic-memory - Ego4D Episodic Memory Benchmark

Ego4D Episodic Memory Benchmark EGO4D is the world's largest egocentric (first p

null 3 Feb 18, 2022
FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

FuseDream This repo contains code for our paper (paper link): FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizat

XCL 191 Dec 31, 2022
DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time Introduction This is official implementation for DR-GAN (IEEE TCS

Kang Liao 18 Dec 23, 2022
Discovering Interpretable GAN Controls [NeurIPS 2020]

GANSpace: Discovering Interpretable GAN Controls Figure 1: Sequences of image edits performed using control discovered with our method, applied to thr

Erik Härkönen 1.7k Jan 3, 2023
[NeurIPS 2021] Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data

Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data (NeurIPS 2021) This repository provides the official PyTorch implementation

Liming Jiang 155 Nov 30, 2021
PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement.

DECOR-GAN PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement, Zhiqin Chen, Vladimir G. Kim, Matthew Fish

Zhiqin Chen 72 Dec 31, 2022
Pytorch implementation of CVPR2021 paper "MUST-GAN: Multi-level Statistics Transfer for Self-driven Person Image Generation"

MUST-GAN Code | paper The Pytorch implementation of our CVPR2021 paper "MUST-GAN: Multi-level Statistics Transfer for Self-driven Person Image Generat

TianxiangMa 46 Dec 26, 2022
Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

RecycleD Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN

Yunan Zhu 23 Nov 5, 2022
Official and maintained implementation of the paper "OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data" [BMVC 2021].

OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data Christoph Reich, Tim Prangemeier, Özdemir Cetin & Heinz Koeppl | Pr

Christoph Reich 23 Sep 21, 2022
Implementation of TransGanFormer, an all-attention GAN that combines the finding from the recent GanFormer and TransGan paper

TransGanFormer (wip) Implementation of TransGanFormer, an all-attention GAN that combines the finding from the recent GansFormer and TransGan paper. I

Phil Wang 146 Dec 6, 2022
Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equations

ODE GAN (Prototype) in PyTorch Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary

Somshubra Majumdar 15 Feb 10, 2022
Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis

Chunked Autoregressive GAN (CARGAN) Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis [paper] [compan

Descript 150 Dec 6, 2022
Unofficial implementation of HiFi-GAN+ from the paper "Bandwidth Extension is All You Need" by Su, et al.

HiFi-GAN+ This project is an unoffical implementation of the HiFi-GAN+ model for audio bandwidth extension, from the paper Bandwidth Extension is All

Brent M. Spell 134 Dec 30, 2022
Official pytorch implementation of Rainbow Memory (CVPR 2021)

Rainbow Memory: Continual Learning with a Memory of Diverse Samples

Clova AI Research 91 Dec 17, 2022
Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Hierarchical Transformer Memory (HTM) - Pytorch Implementation of Hierarchical Transformer Memory (HTM) for Pytorch. This Deepmind paper proposes a si

Phil Wang 63 Dec 29, 2022