Probabilistic Programming and Statistical Inference in PyTorch

Stefano Peluchetti

Last update: Nov 26, 2022

Related tags

Overview

PtStat

Probabilistic Programming and Statistical Inference in PyTorch.

Introduction

This project is being developed during my time at Cogent Labs.

The documentation is still WIP, a brief API description is reported below. The tests might also be helpful.

The API might change quickly during this initial development period.

API

The first dimension is the batch dimension, over which the samples are assumed to be independent.

# Random variables interface:

class RandomVariable:
    def size(self)        # --> (batch_size, rv_dimension)
        
    def log_pdf(self, x)  # --> [batch_size]

    def sample(self)      # --> [batch_size, rv_dimension]

    def entropy(self)     # --> [batch_size]


# Implemented random variables:

Normal(size=(batch_size, rv_dimension), cuda=cuda)
Normal(mu, sd)

Categorical(size=(batch_size, rv_dimension), cuda=cuda)
Categorical(p)

Bernoulli(size=(batch_size, rv_dimension), cuda=cuda)
Bernoulli(p)

Uniform(size=(batch_size, rv_dimension), cuda=cuda)

# KL-Divergence:

def kld(rv_from, rv_to)  # --> [batch_size]

Changelog

Version 0.2.0

removed specialized distributions => more flexible constructors
refactoring: distributions into multiple files

Version 0.1.0

initial commit

Licensing

The code is released under the MIT license.

Comments

Code review
Hello! I found your repo in pytorch discourse. I'm a pymc3 team member. Very happy that Bayes goes to pytorch too. We were planning to investigate if pytorch is suitable for bayesian reasoning. You can read our discussion here. I'd like to leave some comments for future improvements

Assertion errors can be hard to catch and if you have one, you can't figure out what type is it. I think some of assertions are not needed as they are checked at runtime without any supplementary code.

Now I see all stuff in code. Hope in future it will be more organized. Distribution submodule with splitting to distribution types can be a good solution.

Use pytest. I got used to unittest until tried pytest, now I have no way back to unittest, pytest is awesome

Use docstrings instead of # comment. It can be helpfull for future users

Remove NormalUnit and Uniform01, add defaults instead. Having lots of classes of the same distribution can be messy
opened by ferrine 2

Data-depth-inference - Data depth inference with python

Welcome! This readme will guide you through the use of the code in this reposito

3 Feb 8, 2022

Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

PyTorch implementation of 'Denoising Diffusion Probabilistic Models' This reposi

76 Jan 7, 2023

Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Noise Contrastive Estimation for pyTorch Overview This repository contains a re-implementation of the Noise Contrastive Estimation algorithm, implemen

42 Nov 24, 2022

Official PyTorch implementation for FastDPM, a fast sampling algorithm for diffusion probabilistic models

Official PyTorch implementation for "On Fast Sampling of Diffusion Probabilistic Models". FastDPM generation on CIFAR-10, CelebA, and LSUN datasets. S

68 Dec 26, 2022

Pytorch implementation of "Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech"

GradTTS Unofficial Pytorch implementation of "Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech" (arxiv) About this repo This is an unoffic

103 Dec 23, 2022

Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in Pytorch

Retrieval-Augmented Denoising Diffusion Probabilistic Models (wip) Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in P

55 Jan 1, 2023

Statsmodels: statistical modeling and econometrics in Python

About statsmodels statsmodels is a Python package that provides a complement to scipy for statistical computations including descriptive statistics an

8.1k Jan 2, 2023

Cockpit is a visual and statistical debugger specifically designed for deep learning.

Cockpit: A Practical Debugging Tool for Training Deep Neural Networks

421 Dec 29, 2022

PyStan, a Python interface to Stan, a platform for statistical modeling. Documentation: https://pystan.readthedocs.io

PyStan NOTE: This documentation describes a BETA release of PyStan 3. PyStan is a Python interface to Stan, a package for Bayesian inference. Stan® is

229 Dec 29, 2022

Probabilistic Programming and Statistical Inference in PyTorch

Related tags

Overview

PtStat

Introduction

API

Changelog

Version 0.2.0

Version 0.1.0

Licensing

You might also like...

Data-depth-inference - Data depth inference with python

Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Official PyTorch implementation for FastDPM, a fast sampling algorithm for diffusion probabilistic models

Pytorch implementation of "Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech"

Implementation of Retrieval-Augmented Denoising Diffusion Probabilistic Models in Pytorch

Statsmodels: statistical modeling and econometrics in Python

Cockpit is a visual and statistical debugger specifically designed for deep learning.

PyStan, a Python interface to Stan, a platform for statistical modeling. Documentation: https://pystan.readthedocs.io

Comments

Code review

Owner

Stefano Peluchetti

PyTorch-LIT is the Lite Inference Toolkit (LIT) for PyTorch which focuses on easy and fast inference of large models on end-devices.

Deep universal probabilistic programming with Python and PyTorch

Code to run experiments in SLOE: A Faster Method for Statistical Inference in High-Dimensional Logistic Regression.

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Modular Probabilistic Programming on MXNet

PClean: A Domain-Specific Probabilistic Programming Language for Bayesian Data Cleaning

Deep Probabilistic Programming Course @ DIKU

DeepProbLog is an extension of ProbLog that integrates Probabilistic Logic Programming with deep learning by introducing the neural predicate.

Torchserve server using a YoloV5 model running on docker with GPU and static batch inference to perform production ready inference.

Monocular 3D pose estimation. OpenVINO. CPU inference or iGPU (OpenCL) inference.