Python code for "Machine learning: a probabilistic perspective" (2nd edition)

Overview

pyprobml

Python 3 code for my new book series Probabilistic Machine Learning. This is work in progress, so expect rough edges. Getting less rough...

Jupyter notebooks

For each chapter there are one or more accompanying Jupyter notebooks that cover some of the material in more detail. When you open a notebook, there will be a button at the top that says 'Open in colab'. If you click on this, it will start a virtual machine (VM) instance on Google Cloud Platform (GCP), running Colab. This has most of the libraries you will need (e.g., scikit-learn, JAX) pre-installed, and gives you access to a free GPU. See this tutorial for details on how to use Colab.

Book 1 (PML: An Introduction)

See this link for a list of notebooks.

Book 2 (PML: Advanced topics)

See this link for a list of notebooks.

Running scripts to make individual figures

Many of the figures in the book are generated by various scripts. To run these, first clone this gihub repo. (For some tutorials on how to use github, see github guides.) Then, to manually execute an individual script from the command line, follow this example:

export PYPROBML=/Users/kpmurphy/github/pyprobml // set this to the directory where you downloaded this repo
cd $PYPROBML
python3 scripts/softmax_plot.py // writes to /Users/kpmurphy/github/pyprobml/figures/softmax_temp.pdf

The notebook for each chapter uses these scripts to recreate all the figures for that chapter.

Viewing the scripts

To browse the code using VScode instead of the gihub file viewer, you can just replace https://github.com/probml/pyprobml/tree/master/scripts with https://github1s.com/probml/pyprobml/tree/master/scripts (see this tweet). The output should look like this:

How to contribute

See this guide for how to contribute code.

Acknowledgements

I would like to thank the following people for contributing to the code (list autogenerated from this page):

murphyk mjsML ashishpapanai Duane321 animesh-007 Nirzu97 Drishttii always-newbie161 nappaillav jdf22 shivaditya-meduri karalleyna gerdm andrewnc Abdelrahman350 Garvit9000c kzymgch Neoanarika alen1010 adamnemecek galv krasserm nealmcb petercerno Prahitha khanshehjad hieuza jlh2018 mvervuurt TripleTop
murphyk mjsML ashishpapanai Duane321 animesh-007 Nirzu97 Drishttii always-newbie161 nappaillav jdf22 shivaditya-meduri karalleyna gerdm andrewnc Abdelrahman350 Garvit9000c kzymgch Neoanarika alen1010 adamnemecek galv krasserm nealmcb petercerno Prahitha khanshehjad hieuza jlh2018 mvervuurt TripleTop
Comments
  • convert legacy matlab code to python

    convert legacy matlab code to python

    If you want to contribute a Python version of a figure, follow these steps:

    • Check the list of open issues for ones with a name like "Convert foo to Python". Such issues have the tag 'Figures'. Each issue has a link to the Matlab file that was used to generate the figure, and a figure number. Figure numbers refer to this version of the book: https://github.com/probml/pml-book/releases/tag/2021-03-08. Consult the pdf file to see what the figure should look like.

    • Follow these guidelines for how to contribute code.

    • Test your code works in Google colab, and make a pull request. Please include the generated image(s) in your PR, and add a tag which mentions the issue you are solving.

    • Here is an example of good PR.

    • If you are a student and want to be considered for the Google Summer of Code, please apply here after March 29th. You should have at least 2 PRs succesfully added to this repo before applying. Please follow the style guidelines mentioned above.

    opened by murphyk 25
  • convert pytorch d2l.ai demos to JAX

    convert pytorch d2l.ai demos to JAX

    In vol 1 of my book, I use several notebooks taken from the book Dive Into Deep Learning. These are written in pytorch and stored at https://github.com/probml/probml-notebooks/tree/main/notebooks-d2l. The task is to convert these notebooks into JAX/FLAX code. It's okay to keep using torch dataloaders, but you should replace torch.tensor with jax.np.array, and torch.nn with flax.linen.nn, etc. Be sure to replace idioms like np.random.seed(seed=1); torch.manual_seed(1) with proper JAX PRNGs.

    You should make a JAX version of each notebook, and open a PR for each one at https://github.com/probml/probml-notebooks/tree/main/notebooks-d2l. If the original notebook is called foo_torch.ipynb, please call yours foo_jax.ipynb.

    See this example for how to format your code and PR.

    Before you start translating a notebook, please check that it has not already been done! (I only want one translation per notebook :)

    GSOC 
    opened by murphyk 17
  • Accelerate workflow by including SMOKE_RUN

    Accelerate workflow by including SMOKE_RUN

    After adding D2L notebooks, workflow execution time has increased from 1 hour to 2 hours 14 minutes. Ideally, the workflow should be as fast as possible so that we can know the failing components quickly. One of the solutions for this is to enable an environment variable called SMOKE_RUN for notebooks containing epoch-wise training with a logic that if SMOKE_RUN is in the environment we run only 1-2 epochs else we run the maximum epochs as mentioned in the notebook.

    internal 
    opened by patel-zeel 15
  • mlpPriorsDemo2

    mlpPriorsDemo2

    Hello, this is a Python implementation for mlpPriorsDemo2. The figures are slightly different from the book because NumPy generates different random vectors than Matlab.

    opened by Abdelrahman350 14
  • figure out why ELBO does not improve with pymc3 ADVI demi

    figure out why ELBO does not improve with pymc3 ADVI demi

    The advi.hist (ELBO) should increase over time, but it does not seem to, even though the posterior looks sensible. Find out why. https://github.com/probml/probml-notebooks/blob/main/notebooks/beta_binom_approx_post_pymc.ipynb

    GSOC 
    opened by murphyk 13
  • Porting the complete book code to Python?

    Porting the complete book code to Python?

    Hello Kevin,

    I was wondering if there is a plan to port all of the code examples from the ML book to Python?

    If that's the case I'd love to help out :)

    opened by thvasilo 12
  • Implemented SNGP JAX Demo

    Implemented SNGP JAX Demo

    opened by nsanghi 11
  • implement blocked gibbs sampling for Bayesian inference in linear Gaussian SSM  using JSL

    implement blocked gibbs sampling for Bayesian inference in linear Gaussian SSM using JSL

    Implement blocked gibbs sampling for Bayesian inference in linear Gaussian SSM . In the "E step", use the Jax SSM library for forwards-filtering backwards-sampling. In the "M step", sample from the parameter posteriors assuming conjugate priors.

    Some details can be found in this paper A. Wills, T. B. Schön, F. Lindsten, and B. Ninness, “Estimation of Linear Systems using a Gibbs Sampler,” IFAC proc. vol., vol. 45, no. 16, pp. 203–208, Jul. 2012, doi: 10.3182/20120711-3-be-2027.00297. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S1474667015379520

    JSL 
    opened by murphyk 10
  • Missing superimport module for compare_results.ipynb

    Missing superimport module for compare_results.ipynb

    I encounter the following error when running compare_results.ipynb in the vae folder.

    ---------------------------------------------------------------------------
    ModuleNotFoundError                       Traceback (most recent call last)
    <ipython-input-2-d04a2cc14755> in <module>()
          5 import matplotlib.pyplot as plt
          6 import torchvision.transforms as transforms
    ----> 7 from download_celeba import celeba_dataloader
          8 from assembler import get_config, assembler
          9 from utils.plot import plot_reconstruction, plot_samples
    
    /content/download_celeba.py in <module>()
    ----> 1 import superimport
          2 
          3 from absl import app
          4 from absl import flags
          5 import torchvision.transforms as transforms
    

    I think this can be resolved by adding one line in the notebook to download superimport into the colab.

    opened by Neoanarika 10
  • Converted mixBerMnistEM to python

    Converted mixBerMnistEM to python

    mixBernoulliMnist

    Closes #174

    The code might take some time to load and this is because it is downloading the MNIST dataset from online and the delay in execution is solely due to this and not due to the inefficiency of the code.

    The values and the images will differ because the data is randomly chosen from the total set of images.

    I initially added +1 to all images in the training set and changed the Bernoulli function to adapt to this, but did not see any change in the execution, It was more convenient for me to use the binary image as is and use the implementation of Bernoulli distribution which is used for this mixture model.

    In this implementation, I did not find the optimal k because the optimal k=20 has already been found through trial and error in the MATLAB code and no graph implementation is found in the book.

    Implemented the changes suggested:

    1. Vectorized code and replaced iterating over the data points to vectorized versions
    2. replaced Bernoulli function with https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.bernoulli.html
    3. factored out the code that does parameter initialization and image plotting into subfunctions, to simplify main()
    4. changed the data source from open_ml to tensorflow.datasets
    opened by shivaditya-meduri 10
  • estimate mutual information for all pairs of WHO variables

    estimate mutual information for all pairs of WHO variables

    opened by murphyk 10
  • Figure 6.1 missing sub-figure c) showing a sequence logo.

    Figure 6.1 missing sub-figure c) showing a sequence logo.

    Figure 6.1 b) shows a graphic depiction of a position frequency matrix, but it is not a sequence logo because all column heights are equal. My guess is the original intent was to have a sub-figure 6.1 c) showing a sequence logo and compare that to the graphic in sub-figure 6.1 b) so the reader can see how much easier it is to spot conserved positions when the column heights are scaled to equal the information content. The text corresponding to figure 6.1 does mention heights scaled to information content (although "bar" height is used instead of "column" height.

    opened by paulhorton 0
  • tweak book 1 fog 7.6  gaussEvec.ipynb

    tweak book 1 fog 7.6 gaussEvec.ipynb

    To solve https://github.com/probml/pml-book/issues/440 Tweak https://colab.research.google.com/github/probml/pyprobml/blob/master/notebooks/book1/07/gaussEvec.ipynb The exponents should be lambda^{-1/2}. I tried changing the text but the figure no longer renders properly (u_1, u_2 axis broken)

    Screen Shot 2022-07-28 at 11 55 07 PM
    opened by murphyk 0
  • Make bnn_mnist_sgld_blackjax.ipynb to be able to run in colab

    Make bnn_mnist_sgld_blackjax.ipynb to be able to run in colab

    • currently, this notebook won't run in colab, due to requirement of high memory.
    • Mostly prediction loops taking much memory, we can make it batch prediction to consume less memory at a time
    opened by karm-patel 0
Releases(v0.1.0)
  • v0.1.0(May 5, 2022)

Owner
Probabilistic machine learning
Material to accompany the book "Machine Learning: A Probabilistic Perspective" (Software, Data, Exercises, Figures, etc)
Probabilistic machine learning
A Django app that creates automatic web UIs for Python scripts.

Wooey is a simple web interface to run command line Python scripts. Think of it as an easy way to get your scripts up on the web for routine data anal

Wooey 1.9k Jan 1, 2023
Freqtrade is a free and open source crypto trading bot written in Python

Freqtrade is a free and open source crypto trading bot written in Python. It is designed to support all major exchanges and be controlled via Telegram. It contains backtesting, plotting and money management tools as well as strategy optimization by machine learning.

null 20.2k Jan 2, 2023
A high-level app and dashboarding solution for Python

Panel provides tools for easily composing widgets, plots, tables, and other viewable objects and controls into custom analysis tools, apps, and dashboards.

HoloViz 2.5k Jan 3, 2023
Python books free to read online or download

Python books free to read online or download

Paolo Amoroso 3.7k Jan 8, 2023
PyMMO is a Python-based MMO game framework using sockets and PyGame.

PyMMO is a Python framework/template of a MMO game built using PyGame on top of Python's built-in socket module.

Luis Souto Maior 61 Dec 18, 2022
A python application for manipulating pandas data frames from the comfort of your web browser

A python application for manipulating pandas data frames from the comfort of your web browser. Data flows are represented as a Directed Acyclic Graph, and nodes can be ran individually as the user sees fit.

Schlerp 161 Jan 4, 2023
A modern Python package manager with PEP 582 support.

A modern Python package manager with PEP 582 support.

Python Development Master(PDM) 3.6k Jan 5, 2023
Python Crypto Bot

Python Crypto Bot

Michael Whittle 1.6k Jan 6, 2023
Firebase Admin Console is a centralized platform for easy viewing and maintenance of Firestore database, the back-end API is a Python Flask app.

Firebase Admin Console is a centralized platform for easy viewing and maintenance of Firestore database, the back-end API is a Python Flask app. A starting template for developers to customize, build, and even deploy the desired admin console for their DB.

Daqi Chen 1 Sep 10, 2022
Simple Linear 2nd ODE Solver GUI - A 2nd constant coefficient linear ODE solver with simple GUI using euler's method

Simple_Linear_2nd_ODE_Solver_GUI Description It is a 2nd constant coefficient li

:) 4 Feb 5, 2022
Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Packt 1.5k Jan 3, 2023
Solutions of Reinforcement Learning 2nd Edition

Solutions of Reinforcement Learning, An Introduction

YIFAN WANG 1.4k Dec 30, 2022
This is the code repository for Mastering Python for Networking and Security – Second Edition

Mastering Python for Networking and Security – Second Edition This is the code repository for Mastering Python for Networking and Security – Second Ed

Frank Gottinger 1 Feb 9, 2022
This is 2nd term discrete maths project done by UCU students that uses backtracking to solve various problems.

Backtracking Project Sponsors This is a project made by UCU students: Olha Liuba - crossword solver implementation Hanna Yershova - sudoku solver impl

Dasha 4 Oct 17, 2021
2nd solution of ICDAR 2021 Competition on Scientific Literature Parsing, Task B.

TableMASTER-mmocr Contents About The Project Method Description Dependency Getting Started Prerequisites Installation Usage Data preprocess Train Infe

Jianquan Ye 298 Dec 21, 2022
Kaggle G2Net Gravitational Wave Detection : 2nd place solution

Kaggle G2Net Gravitational Wave Detection : 2nd place solution

Hiroshechka Y 33 Dec 26, 2022
Our Ping Pong Project of numerical analysis, 2nd year IC B2 INSA Toulouse

Ping Pong Project The objective of this project was to determine the moment of impact of the ball with the ground. To do this, we used different model

null 0 Jan 2, 2022
The 2nd Version Of Slothybot

SlothyBot Go to this website: "https://bitly.com/SlothyBot" The 2nd Version Of Slothybot. The Bot Has Many Features, Such As: Moderation Commands; Kic

Slothy 0 Jun 1, 2022
Xview3 solution - XView3 challenge, 2nd place solution

Xview3, 2nd place solution https://iuu.xview.us/ test split aggregate score publ

Selim Seferbekov 24 Nov 23, 2022
This is the solution for 2nd rank in Kaggle competition: Feedback Prize - Evaluating Student Writing.

Feedback Prize - Evaluating Student Writing This is the solution for 2nd rank in Kaggle competition: Feedback Prize - Evaluating Student Writing. The

Udbhav Bamba 41 Dec 14, 2022