Torch-based tool for quantizing high-dimensional vectors using additive codebooks

Daniel Povey

Last update: Jan 7, 2023

Related tags

Deep Learning quantization

Overview

Trainable multi-codebook quantization

This repository implements a utility for use with PyTorch, and ideally GPUs, for training an efficient quantizer based on multiple single-byte codebooks. The prototypical scenario is that you have some distribution over vectors in some space, say, of dimension 512, that might come from a neural net embedding, and you want a means of encoding a vector into a short sequence of bytes (say, 4 or 8 bytes) that can be used to reconstruct the vector with minimal expected loss, measured as squared distance, i.e. squared l2 loss.

This repository provides Quantizer object that lets you do this quantization, and an associated QuantizerTrainer object that you can use to train the Quantizer. For example, you might invoke the QuantizerTrainer with 20,000 minibatches of vectors.

Usage

Installation

python3 setup.py install

Example

import torch
import quantization

trainer = quantization.QuantizerTrainer(dim=256, bytes_per_frame=4,
                                        device=torch.device('cuda'))
while not trainer.done():
   # let x be some tensor of shape (*, dim), that you will train on
   # (should not be the same on each minibatch)
   trainer.step(x)
quantizer = trainer.get_quantizer()

# let x be some tensor of shape (*, dim)..
encoded = quantizer.encode(x)  # (*, 4), dtype=uint8
x_approx = quantizer.decode(quantizer.encode(x))

To avoid versioning issues and so on, it may be easier to just include quantization.py in your repository directly (and add its requirements to your requirements.txt).

You might also like...

Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch

Automatic Number Plate Recognition Automatic Number Plate Recognition (ANPR) is the process of reading the characters on the plate with various optica

52 Dec 22, 2022

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayesian-Torch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.

210 Jan 4, 2023

Simple torch.nn.module implementation of Alias-Free-GAN style filter and resample

Alias-Free-Torch Simple torch module implementation of Alias-Free GAN. This repository including Alias-Free GAN style lowpass sinc filter @filter.py A

64 Dec 22, 2022

A machine learning library for spiking neural networks. Supports training with both torch and jax pipelines, and deployment to neuromorphic hardware.

Rockpool Rockpool is a Python package for developing signal processing applications with spiking neural networks. Rockpool allows you to build network

21 Dec 14, 2022

Pytorch Implementations of large number classical backbone CNNs, data enhancement, torch loss, attention, visualization and some common algorithms.

Torch-template-for-deep-learning Pytorch implementations of some **classical backbone CNNs, data enhancement, torch loss, attention, visualization and

270 Dec 31, 2022

Comments

fix_circular_import

These modifications solve "circular import" error when installed by command:

pip install git+https://github.com/danpovey/quantization.git@master
python -c "import quantization"

Error message:

from quantization import Quantizer
ImportError: cannot import name 'Quantizer' from partially initialized module 'quantization' (most likely due to a circular import) (/ceph-ly/env/torchvi/lib/python3.8/site-packages/quantization/__init__.py)

opened by glynpu 0

Prediction

This branch adds a module which is a convenient way to predict these codebook entries, regressing on something. (In the test code, we regress on the vector itself, that we encoded, but in general it would be something else that we are trying to learn).

This supports also regressing on previous codebook entries in the list of codebooks, so in effect it is jointly predicting the whole set of codebook entries, rather than modeling them independently. We may find that this improves results.

opened by danpovey 0

Torch-based tool for quantizing high-dimensional vectors using additive codebooks

Related tags

Overview

Trainable multi-codebook quantization

Usage

Installation

Example

You might also like...

Automatic number plate recognition using tech: Yolo, OCR, Scene text detection, scene text recognation, flask, torch

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Simple torch.nn.module implementation of Alias-Free-GAN style filter and resample

A machine learning library for spiking neural networks. Supports training with both torch and jax pipelines, and deployment to neuromorphic hardware.

A torch.Tensor-like DataFrame library supporting multiple execution runtimes and Arrow as a common memory format

Pytorch and Torch testing code of CartoonGAN

Implements Stacked-RNN in numpy and torch with manual forward and backward functions

Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution"

Pytorch Implementations of large number classical backbone CNNs, data enhancement, torch loss, attention, visualization and some common algorithms.

Comments

fix_circular_import

Prediction

Owner

Daniel Povey

Unofficial Tensorflow-Keras implementation of Fastformer based on paper [Fastformer: Additive Attention Can Be All You Need](https://arxiv.org/abs/2108.09084).

Neural network-based build time estimation for additive manufacturing

Vector AI — A platform for building vector based applications. Encode, query and analyse data using vectors.

Unofficial PyTorch implementation of Neural Additive Models (NAM) by Agarwal, et al.

An implementation of Fastformer: Additive Attention Can Be All You Need in TensorFlow

Code to run experiments in SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression.

The implementation for paper Joint t-SNE for Comparable Projections of Multiple High-Dimensional Datasets.

Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

Code for paper PairRE: Knowledge Graph Embeddings via Paired Relation Vectors.