A TensorFlow implementation of elastic weight consolidation as presented in Overcoming catastrophic forgetting in neural networks.
Perform hyperparameter search over learning rates for the permuted MNIST task (fisher multiplier locked at inverse learning rate):
python -u main.py --hidden_layers 2 --hidden_units 800 --num_perms 5 --trials 50 --epochs 100
Learning Associative Inference Using Fast Weight Memory This repository contains the offical code for the paper Learning Associative Inference Using F
AnimDL - Download & Stream Your Favorite Anime AnimDL is an incredibly powerful tool for downloading and streaming anime. Core features Abuses the dev
An integration of several popular automatic augmentation methods, including OHL (Online Hyper-Parameter Learning for Auto-Augmentation Strategy) and AWS (Improving Auto Augment via Augmentation Wise Weight Sharing) by Sensetime Research.
Recurrent Fast Weight Programmers This is the official repository containing the code we used to produce the experimental results reported in the pape
An image labelling tool for creating segmentation data sets, for Django and Flask.
DeLighT: Very Deep and Light-weight Transformers This repository contains the source code of our work on building efficient sequence models: DeFINE (I
Simpler is Better: Few-shot Semantic Segmentation with Classifier Weight Transformer. ICCV2021. Introduction We proposed a novel model training paradi
Note: this repo has been discontinued, please check code for newer version of the paper here Weight Normalized GAN Code for the paper "On the Effects
Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning Overview This code is for paper: Not All Unlabeled Data are Equa
As I understand it, the fisher weights of x for weights are the squares of the gradients d/dw (log x). (Let me know if this is incorrect.)
In my own implementations I use the loss term for x, but in other implementations I see the model prediction used instead. I can't remember why I used this, although I suspect it's because I needed a value >0 (for which raw predictions of a Q-network would not work).
Should the Fisher matrix be based on the raw output rather than the error? (In practice, it seems to work based on the error, which makes sense, although I suspect basing it on the output would work better.)
I am not sure i understand correctly.
it looks like you are compute fisher matrix for current task with samples after the training a current task. am i right?
Then, when moving to third task... does fisher diagonal compute for all previous tasks? or just compute previous task only? I think if the former, EWC is useless because we do not want to see previous. right?
i also implement the EWC using tensorflow. but, i am not sure that i am right. so i am referring your code :D. Thank you for your advanced work!
p.s why do we need to compute fisher diagonal per example? why not per batch?
CONVERT YOUR MODEL TO IR FORMAT INSTALLATION OpenVino Toolkit Download openvinotoolkit 2021.3 version : Link Instruction of installation : Link Pytorc
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient (paper) @misc{zhang2021compress,
ElasticGNN This repository includes the official implementation of ElasticGNN in the paper "Elastic Graph Neural Networks" [ICML 2021]. Xiaorui Liu, W
M-LSD: Towards Light-weight and Real-time Line Segment Detection Official Tensorflow implementation of "M-LSD: Towards Light-weight and Real-time Line
H2O H2O is an in-memory platform for distributed, scalable machine learning. H2O uses familiar interfaces like R, Python, Scala, Java, JSON and the Fl
One-Shot Voice Conversion with Weight Adaptive Instance Normalization By Shengjie Huang, Yanyan Xu*, Dengfeng Ke*, Mingjie Chen, Thomas Hain. This rep
M-LSD: Towards Light-weight and Real-time Line Segment Detection Pytorch implementation of "M-LSD: Towards Light-weight and Real-time Line Segment Det
AceNAS This repo is the experiment code of AceNAS, and is not considered as an official release. We are working on integrating AceNAS as a built-in st
MobileViT RegNet Unofficial PyTorch implementation of MobileViT based on paper MOBILEVIT: LIGHT-WEIGHT, GENERAL-PURPOSE, AND MOBILE-FRIENDLY VISION TR
Differentiable Model Compression via Pseudo Quantization Noise DiffQ performs differentiable quantization using pseudo quantization noise. It can auto
18 Oct 12, 2022
759 Jan 8, 2023
45 Dec 8, 2022
36 Nov 15, 2022
117 Nov 21, 2022
440 Dec 18, 2022
103 Dec 14, 2022
182 Sep 6, 2021
22 Nov 23, 2022
3 Nov 18, 2021
46 Dec 7, 2022
34 Dec 4, 2022
357 Jan 4, 2023
6.1k Jan 5, 2023
31 Dec 7, 2022
123 Jan 4, 2023
6 Sep 7, 2022
91 Dec 2, 2022
145 Dec 30, 2022