Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Last update: Nov 3, 2022

Related tags

Overview

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

This repository is the official implementation of paper [Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples].

Requirements

Python 3.6
PyTorch 1.8.1
Refer requirements.txt for other requirements

To install requirements:

pip install -r requirements.txt

Training

For Imagenet training, change the path of the validation set in .hocon file. To train the model described in the paper, run one of this command:

./run_cifar10_4bit.sh
./run_cifar100_4bit.sh
./run_imgnet_resnet18_4bit.sh
./run_imgnet_resnet50_4bit.sh
./run_imgnet_mobilenet_v2_4bit.sh

Refer other_train_scripts folder for 5bit settings.

Evaluation

To evaluate trained model, run the command below after training:

./eval_cifar10_4bit.sh
./eval_cifar100_4bit.sh
./eval_imgnet_resnet18_4bit.sh
./eval_imgnet_resnet50_4bit.sh
./eval_imgnet_mobilenet_v2_4bit.sh

Visualizing Feature Space

Feature space visualization of real or synthetic images described in Figure 3.

python experiments.py --pca_source
python experiments.py --gdfq_generator_path GENERATOR_WEIGHT_PATH --pca_gdfq --image_gdfq
python experiments.py --qimera_generator_path GENERATOR_WEIGHT_PATH --pca_qimera --pca_mix --pca_path --image_gdfq --image_mix

Results

Our model achieves the following performance on :

Dataset	Model	Teacher Net Accuracy	4bit Quantized Model Accuracy	5bit Quantized Model Accuracy
Cifar-10	ResNet-20	93.89%	91.26%	93.46%
Cifar-100	ResNet-20	70.33%	65.10%	69.02%
ImageNet	ResNet-18	71.47%	63.84%	69.29%
ImageNet	ResNet-50	77.73%	66.25%	75.32%
ImageNet	MobileNetV2	73.03%	61.62%	70.45%

Generated Synthetic Images for Cifar10 :

License

This project is licensed under the terms of the GNU General Public License v3.0

Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

Real-ESRGAN Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data Ported from https://github.com/xinntao/Real-ESRGAN Depend

44 Dec 27, 2022

Shape Matching of Real 3D Object Data to Synthetic 3D CADs (3DV project @ ETHZ)

Real2CAD-3DV Shape Matching of Real 3D Object Data to Synthetic 3D CADs (3DV project @ ETHZ) Group Member: Yue Pan, Yuanwen Yue, Bingxin Ke, Yujie He

24 Jun 22, 2022

Official repository of the paper Privacy-friendly Synthetic Data for the Development of Face Morphing Attack Detectors

SMDD-Synthetic-Face-Morphing-Attack-Detection-Development-dataset Official repository of the paper Privacy-friendly Synthetic Data for the Development

10 Dec 12, 2022

existing and custom freqtrade strategies supporting the new hyperstrategy format.

freqtrade-strategies Description Existing and self-developed strategies, rewritten to support the new HyperStrategy format from the freqtrade-develop

39 Aug 20, 2021

Pytorch implementation of Supporting Clustering with Contrastive Learning, NAACL 2021

Supporting Clustering with Contrastive Learning SCCL (NAACL 2021) Dejiao Zhang, Feng Nan, Xiaokai Wei, Shangwen Li, Henghui Zhu, Kathleen McKeown, Ram

231 Jan 5, 2023

Supporting code for the Neograd algorithm

Neograd This repo supports the paper Neograd: Gradient Descent with a Near-Ideal Learning Rate, which introduces the algorithm "Neograd". The paper an

12 May 1, 2022

Supporting code for the paper "Dangers of Bayesian Model Averaging under Covariate Shift"

Dangers of Bayesian Model Averaging under Covariate Shift This repository contains the code to reproduce the experiments in the paper Dangers of Bayes

25 Sep 21, 2022

Implementation supporting the ICCV 2017 paper "GANs for Biological Image Synthesis"

GANs for Biological Image Synthesis This codes implements the ICCV-2017 paper "GANs for Biological Image Synthesis". The paper and its supplementary m

95 Nov 25, 2022

A torch.Tensor-like DataFrame library supporting multiple execution runtimes and Arrow as a common memory format

TorchArrow (Warning: Unstable Prototype) This is a prototype library currently under heavy development. It does not currently have stable releases, an

536 Jan 6, 2023

Comments

run_imgnet_resnet18_4bit.sh get the acc？

HI，Thank you for your contribution, I run the repo run_imgnet_resnet18_4bit.sh,I get the acc: run the eval_imgnet_resnet18_4bit.sh,the result:

but there is big gap for the paper‘s acc 63.84%,How can I reproduce your work？

opened by GuangyanZhang 8
Question about the "generator" module (Not related to GAN model)

Hi, I meet an issue as below ModuleNotFoundError: No module named 'generator'

May I know which generator module by pip or conda that I should install?

opened by he159ok 1

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Related tags

Overview

Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples

Requirements

Training

Evaluation

Visualizing Feature Space

Results

License

You might also like...

Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

Shape Matching of Real 3D Object Data to Synthetic 3D CADs (3DV project @ ETHZ)

Official repository of the paper Privacy-friendly Synthetic Data for the Development of Face Morphing Attack Detectors

existing and custom freqtrade strategies supporting the new hyperstrategy format.

Pytorch implementation of Supporting Clustering with Contrastive Learning, NAACL 2021

Supporting code for the Neograd algorithm

Supporting code for the paper "Dangers of Bayesian Model Averaging under Covariate Shift"

Implementation supporting the ICCV 2017 paper "GANs for Biological Image Synthesis"

A torch.Tensor-like DataFrame library supporting multiple execution runtimes and Arrow as a common memory format

Comments

run_imgnet_resnet18_4bit.sh get the acc？

Question about the "generator" module (Not related to GAN model)

Owner

Kanghyun Choi

DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Quantization library for PyTorch. Support low-precision and mixed-precision quantization, with hardware implementation through TVM.

Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation. In CVPR 2022.

IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization

Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data - Official PyTorch Implementation (CVPR 2022)

Code for CVPR2021 paper "Learning Salient Boundary Feature for Anchor-free Temporal Action Localization"

[IJCAI-2021] A benchmark of data-free knowledge distillation from paper "Contrastive Model Inversion for Data-Free Knowledge Distillation"

Official repo for the work titled "SharinGAN: Combining Synthetic and Real Data for Unsupervised GeometryEstimation"

Code and data of the ACL 2021 paper: Few-Shot Text Ranking with Meta Adapted Synthetic Weak Supervision

Code release for our paper, "SimNet: Enabling Robust Unknown Object Manipulation from Pure Synthetic Data via Stereo"