A booklet on machine learning systems design with exercises

Related tags

Deep Learning machine-learning-systems-design
Overview

Machine Learning Systems Design

Read this booklet here.

This booklet covers four main steps of designing a machine learning system:

  1. Project setup
  2. Data pipeline
  3. Modeling: selecting, training, and debugging
  4. Serving: testing, deploying, and maintaining

It comes with links to practical resources that explain each aspect in more details. It also suggests case studies written by machine learning engineers at major tech companies who have deployed machine learning systems to solve real-world problems.

At the end, the booklet contains 27 open-ended machine learning systems design questions that might come up in machine learning interviews. The answers for these questions will be published in the book Machine Learning Interviews. You can look at and contribute to community answers to these questions on GitHub here. You can read more about the book and sign up for the book's mailing list here.

Contribute

This is work-in-progress so any type of contribution is very much appreciated. Here are a few ways you can contribute:

  1. Improve the text by fixing any lexical, grammatical, or technical error
  2. Add more relevant resources to each aspect of the machine learning project flow
  3. Add/edit questions
  4. Add/edit answers
  5. Other

This book was created using the wonderful magicbook package. For detailed instructions on how to use the package, see their GitHub repo. The package requires that you have node. If you're on Mac, you can install node using:

brew install node

Install magicbook with:

npm install magicbook

Clone this repository:

git clone https://github.com/chiphuyen/machine-learning-systems-design.git
cd machine-learning-systems-design

After you've made changes to the content in the content folder, you can build the booklet by the following steps:

magicbook build

You'll find the generated HTML and PDF files in the folder build.

Acknowledgment

I'd like to thank Ben Krause for being a great friend and helping me with this draft!

Citation

Comments
  • BLEU upper limit 100？

    BLEU upper limit 100？

    @chiphuyen Hi Huyen I think the upper limit of BLEU score in machine translation is not 100. It is some float number between 4-6. I once did an experiment with all reference sentences to be the same as the hypothesis sentence and found the number. Just cannot remember it clearly.

    opened by nkjsy 2
  • Add Netlify configuration

    Add Netlify configuration

    In an effort to make this more accessible, I created a Netlify config file.

    Netlify is a free static site host that will automatically update your site when you push to Github. I forked the repo and created my own netlify site: https://ml-system.netlify.com/.

    If you merge this and create a Netlify account, you can easily create your own site that will update automatically: https://docs.netlify.com/site-deploys/create-deploys/#deploy-with-git.

    opened by tdhopper 1
  • Missing link to contribute to community answers

    Missing link to contribute to community answers

    At the beginning of the exercise section:

    You can look at and contribute to community answers to these questions on GitHub [here].

    the [here] is missing a link.

    Thanks for the great content!

    opened by ThomVett 1
  • Update on dead link - Hyperparameter Optimization

    Update on dead link - Hyperparameter Optimization

    Original Link to the book was: https://www.automl.org/wp-content/uploads/2018/09/chapter1-hpo.pdf which gives a Page was not found. Found this link through a quick search

    opened by kfvincentpan 0
  • Build PDF failed

    Build PDF failed

    I try to build with

    magicbook build

    It print out console like this

    Config file detected: magicbook.json
Prince XML error
-> HTML build finished

    In build folder, there are no PDF(s)

    opened by ttpro1995 1
  • Link doesnt work

    Link doesnt work

    Hi Chip, Thank you for helping the community with wonderful content!

    As for the issues, The link on the page https://huyenchip.com/ml-interviews-book/contents/2.1.1.1-technical-skills.html [link] https://bestpracticer.com/topic/engineering-1521158538386x966527170501649400 is showing page doesn't exist, Maybe the link is wrong?

    opened by Manojbhat09 0
  • A package missing from readme.md

    A package missing from readme.md

    I tried to use this repo of yours and to successfully run the repo, you need the princexml package as well.

    Download link: https://www.princexml.com/download/

    Command to run: sh /path/to/princexml_downloaded_directory

    opened by D3nii 0
Owner
Chip Huyen
Building infra for real-time ML
Chip Huyen
GitHub
Code for Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data? (SDM 2022)

Private Recommender Systems: How Can Users Build Their Own Fair Recommender Systems without Log Data? (SDM 2022) We consider how a user of a web servi

joisino 20 Aug 21, 2022
A repo with study material, exercises, examples, etc for Devnet SPAUTO

MPLS in the SDN Era --> DevNet SPAUTO Get right to the study material: Checkout the Wiki! A lab topology based on MPLS in the SDN era book used for 30

Hugo Tinoco 67 Nov 16, 2022
AI virtual gym is an AI program which can be used to exercise and can be used to see if we are doing the exercises

AI virtual gym is an AI program which can be used to exercise and can be used to see if we are doing the exercises

null 4 Feb 13, 2022
Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.

Machine Learning From Scratch About Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The purpose

Erik Linder-Norén 21.8k Jan 9, 2023
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

This is the Vowpal Wabbit fast online learning code. Why Vowpal Wabbit? Vowpal Wabbit is a machine learning system which pushes the frontier of machin

Vowpal Wabbit 8.1k Jan 6, 2023
Improving Machine Translation Systems via Isotopic Replacement

CAT (Improving Machine Translation Systems via Isotopic Replacement) Machine translation plays an essential role in people’s daily international commu

Zeyu Sun 10 Nov 30, 2022
A Sklearn-like Framework for Hyperparameter Tuning and AutoML in Deep Learning projects. Finally have the right abstractions and design patterns to properly do AutoML. Let your pipeline steps have hyperparameter spaces. Enable checkpoints to cut duplicate calculations. Go from research to production environment easily.

Neuraxle Pipelines Code Machine Learning Pipelines - The Right Way. Neuraxle is a Machine Learning (ML) library for building machine learning pipeline

Neuraxio 555 Dec 24, 2022
Simple and Robust Loss Design for Multi-Label Learning with Missing Labels

Simple and Robust Loss Design for Multi-Label Learning with Missing Labels Official PyTorch Implementation of the paper Simple and Robust Loss Design

Xinyu Huang 28 Oct 27, 2022
GeneDisco is a benchmark suite for evaluating active learning algorithms for experimental design in drug discovery.

GeneDisco is a benchmark suite for evaluating active learning algorithms for experimental design in drug discovery.

null 22 Dec 12, 2022
Implementation of the GVP-Transformer, which was used in the paper "Learning inverse folding from millions of predicted structures" for de novo protein design alongside Alphafold2

GVP Transformer (wip) Implementation of the GVP-Transformer, which was used in the paper Learning inverse folding from millions of predicted structure

Phil Wang 19 May 6, 2022
A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

Master status: Development status: Package information: TPOT stands for Tree-based Pipeline Optimization Tool. Consider TPOT your Data Science Assista

Epistasis Lab at UPenn 8.9k Dec 30, 2022
Scripts of Machine Learning Algorithms from Scratch. Implementations of machine learning models and algorithms using nothing but NumPy with a focus on accessibility. Aims to cover everything from basic to advance.

Algo-ScriptML Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The goal of this project is not t

Algo Phantoms 81 Nov 26, 2022
This is a Machine Learning Based Hand Detector Project, It Uses Machine Learning Models and Modules Like Mediapipe, Developed By Google!

Machine Learning Hand Detector This is a Machine Learning Based Hand Detector Project, It Uses Machine Learning Models and Modules Like Mediapipe, Dev

Popstar Idhant 3 Feb 25, 2022
Code of paper "CDFI: Compression-Driven Network Design for Frame Interpolation", CVPR 2021

CDFI (Compression-Driven-Frame-Interpolation) [Paper] (Coming soon...) | [arXiv] Tianyu Ding*, Luming Liang*, Zhihui Zhu, Ilya Zharkov IEEE Conference

Tianyu Ding 95 Dec 4, 2022
Twins: Revisiting the Design of Spatial Attention in Vision Transformers

Twins: Revisiting the Design of Spatial Attention in Vision Transformers Very recently, a variety of vision transformer architectures for dense predic

null 482 Dec 18, 2022
Generative Models for Graph-Based Protein Design

Graph-Based Protein Design This repo contains code for Generative Models for Graph-Based Protein Design by John Ingraham, Vikas Garg, Regina Barzilay

John Ingraham 159 Dec 15, 2022
Code for "ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on", accepted at WACV 2021 Generation of Human Behavior Workshop.

ShineOn: Illuminating Design Choices for Practical Video-based Virtual Clothing Try-on [ Paper ] [ Project Page ] This repository contains the code fo

Andrew Jong 97 Dec 13, 2022
Code for reproducing our analysis in the paper titled: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency

Image Crop Analysis This is a repo for the code used for reproducing our Image Crop Analysis paper as shared on our blog post. If you plan to use this

Twitter Research 239 Jan 2, 2023
Implementation of paper "DeepTag: A General Framework for Fiducial Marker Design and Detection"

Implementation of paper DeepTag: A General Framework for Fiducial Marker Design and Detection. Project page: https://herohuyongtao.github.io/research/

Yongtao Hu 46 Dec 12, 2022