Responsible Machine Learning with Python

Overview

Responsible Machine Learning with Python

Examples of techniques for training interpretable machine learning (ML) models, explaining ML models, and debugging ML models for accuracy, discrimination, and security.

Overview

Usage of artificial intelligence (AI) and ML models is likely to become more commonplace as larger swaths of the economy embrace automation and data-driven decision-making. While these predictive systems can be quite accurate, they have often been inscrutable and unappealable black boxes that produce only numeric predictions with no accompanying explanations. Unfortunately, recent studies and recent events have drawn attention to mathematical and sociological flaws in prominent weak AI and ML systems, but practitioners don’t often have the right tools to pry open ML models and debug them. This series of notebooks introduces several approaches that increase transparency, accountability, and trustworthiness in ML models. If you are a data scientist or analyst and you want to train accurate, interpretable ML models, explain ML models to your customers or managers, test those models for security vulnerabilities or social discrimination, or if you have concerns about documentation, validation, or regulatory requirements, then this series of Jupyter notebooks is for you! (But please don't take these notebooks or associated materials as legal compliance advice.)

The notebooks highlight techniques such as:

The notebooks can be accessed through:

Further reading:


Enhancing Transparency in Machine Learning Models with Python and XGBoost - Notebook

Monotonicity constraints can turn opaque, complex models into transparent, and potentially regulator-approved models, by ensuring predictions only increase or only decrease for any change in a given input variable. In this notebook, I will demonstrate how to use monotonicity constraints in the popular open source gradient boosting package XGBoost to train an interpretable and accurate nonlinear classifier on the UCI credit card default data.

Once we have trained a monotonic XGBoost model, we will use partial dependence plots and individual conditional expectation (ICE) plots to investigate the internal mechanisms of the model and to verify its monotonic behavior. Partial dependence plots show us the way machine-learned response functions change based on the values of one or two input variables of interest while averaging out the effects of all other input variables. ICE plots can be used to create more localized descriptions of model predictions, and ICE plots pair nicely with partial dependence plots. An example of generating regulator mandated reason codes from high fidelity Shapley explanations for any model prediction is also presented. The combination of monotonic XGBoost, partial dependence, ICE, and Shapley explanations is likely one of the most direct ways to create an interpretable machine learning model today.

Increase Transparency and Accountability in Your Machine Learning Project with Python - Notebook

Gradient boosting machines (GBMs) and other complex machine learning models are popular and accurate prediction tools, but they can be difficult to interpret. Surrogate models, feature importance, and reason codes can be used to explain and increase transparency in machine learning models. In this notebook, we will train a GBM on the UCI credit card default data. Then we’ll train a decision tree surrogate model on the original inputs and predictions of the complex GBM model and see how the variable importance and interactions displayed in the surrogate model yield an overall, approximate flowchart of the complex model’s predictions. We will also analyze the global variable importance of the GBM and compare this information to the surrogate model, our domain expertise, and our reasonable expectations.

To get a better picture of the complex model’s local behavior and to enhance the accountability of the model’s predictions, we will use a variant of the leave-one-covariate-out (LOCO) technique. LOCO enables us to calculate the local contribution each input variable makes toward each model prediction. We will then rank the local contributions to generate reason codes that describe, in plain English, the model’s decision process for every prediction.

Increase Fairness in Your Machine Learning Project with Disparate Impact Analysis using Python and H2O - Notebook

Fairness is an incredibly important, but highly complex entity. So much so that leading scholars have yet to agree on a strict definition. However, there is a practical way to discuss and handle observational fairness, or how your model predictions affect different groups of people. This procedure is often known as disparate impact analysis (DIA). DIA is far from perfect, as it relies heavily on user-defined thresholds and reference levels to measure disparity and does not attempt to remediate disparity or provide information on sources of disparity, but it is a fairly straightforward method to quantify your model’s behavior across sensitive demographic segments or other potentially interesting groups of observations. Some types of DIA are also an accepted, regulation-compliant tool for fair-lending purposes in the U.S. financial services industry. If it’s good enough for multibillion-dollar credit portfolios, it’s probably good enough for your project.

This example DIA notebook starts by training a monotonic gradient boosting machine (GBM) classifier on the UCI credit card default data using the popular open source library, h2o. A probability cutoff for making credit decisions is selected by maximizing the F1 statistic and confusion matrices are generated to summarize the GBM’s decisions across men and women. A basic DIA procedure is then conducted using the information stored in the confusion matrices and some traditional fair lending measures.

Explain Your Predictive Models to Business Stakeholders with LIME using Python and H2O - Notebook

Machine learning can create very accurate predictive models, but these models can be almost impossible to explain to your boss, your customers, or even your regulators. This notebook will use (Local Interpretable Model-agnostic Explanations) LIME to increase transparency and accountability in a complex GBM model trained on the UCI credit card default data. LIME is a method for building linear surrogate models for local regions in a data set, often single rows of data. LIME sheds light on how model predictions are made and describes local model mechanisms for specific rows of data. Because the LIME sampling process may feel abstract to some practitioners, this notebook will also introduce a more straightforward method of creating local samples for LIME.

Once local samples have been generated, we will fit LIME models to understand local trends in the complex model’s predictions. LIME can also tell us the local contribution of each input variable toward each model prediction, and these contributions can be sorted to create reason codes -- plain English explanations of every model prediction. We will also validate the fit of the LIME model to enhance trust in our explanations using the local model’s R2 statistic and a ranked prediction plot.

Testing Machine Learning Models for Accuracy, Trustworthiness, and Stability with Python and H2O - Notebook

Because ML model predictions can vary drastically for small changes in input variable values, especially outside of training input domains, sensitivity analysis is perhaps the most important validation technique for increasing trust in ML model predictions. Sensitivity analysis investigates whether model behavior and outputs remain stable when input data is intentionally perturbed, or other changes are simulated in input data. In this notebook, we will enhance trust in a complex credit default model by testing and debugging its predictions with sensitivity analysis.

We’ll further enhance trust in our model using residual analysis. Residuals refer to the difference between the recorded value of a target variable and the predicted value of a target variable for each row in a data set. Generally, the residuals of a well-fit model should be randomly distributed, because good models will account for most phenomena in a data set, except for random error. In this notebook, we will create residual plots for a complex model to debug any accuracy problems arising from overfitting or outliers.

Machine Learning Model Debugging with Python: All Models are Wrong ... but Why is My Model Wrong? (And Can I Fix It?)

Part 1: Sensitivity Analysis - Notebook

Sensitivity analysis is the perturbation of data under a trained model. It can take many forms and arguably Shapley feature importance, partial dependence, individual conditional expectation, and adversarial examples are all types of sensitivity analysis. This notebook focuses on using these different types of sensitivity analysis to discover error mechanisms and security vulnerabilities and to assess stability and fairness in a trained XGBoost model. It begins by loading the UCI credit card default data and then training an interpretable, monotonically constrained XGBoost model. After the model is trained, global and local Shapley feature importance is calculated. These Shapley values help inform the application of partial dependence and ICE, and together these results guide a search for adversarial examples. The notebook closes by exposing the trained model to a random attack and analyzing the attack results.

These model debugging exercises uncover accuracy, drift, and security problems such as over-emphasis of important features and impactful yet non-robust interactions. Several remediation mechanisms are proposed including editing of final model artifacts to remove or fix errors, missing value injection or regularization during training to lessen the impact of certain features or interactions, and assertion-based missing value injection during scoring to mitigate the effect of non-robust interactions.

Part 2: Residual Analysis - Notebook

In general, residual analysis can be characterized as a careful study of when and how models make mistakes. A better understanding of mistakes will hopefully lead to fewer of them. This notebook uses variants of residual analysis to find error mechanisms and security vulnerabilities and to assess stability and fairness in a trained XGBoost model. It begins by loading the UCI credit card default data and then training an interpretable, monotonically constrained XGBoost gradient boosting machine (GBM) model. (Pearson correlation with the prediction target is used to determine the direction of the monotonicity constraints for each input variable.) After the model is trained, its logloss residuals are analyzed and explained thoroughly and the constrained GBM is compared to a benchmark linear model. These model debugging exercises uncover accuracy, drift, and security problems such as over-emphasis of important variables and strong signal in model residuals. Several remediation mechanisms are proposed including missing value injection during training, additional data collection, and use of assertions to correct known problems during scoring.

From GLM to GBM: Building the Case For Complexity - Notebook

This notebook uses the same credit card default scenario to show how monotonicity constraints, Shapley values and other post-hoc explanations, and discrimination testing can enable practitioners to create direct comparisons between GLM and GBM models. Several candidate probability of default models are selected for comparison using feature selection methods, like LASSO, and by cross-validated ranking. Comparisons then enable building from GLM to more complex GBM models in a step-by-step manner, while retaining model transparency and the ability to test for discrimination. This notebook shows that GBMs can yield better accuracy, more revenue, and that GBMs are also likely to fulfill many model documentation, adverse action notice, and discrimination testing requirements.

Using the Examples

H2O Aquarium (recommended)

H2O Aquarium is a free educational environment that hosts versions of these notebooks among many other H2o-related resources. To use these notebooks in Aquarium:

  1. Navigate to the Aquarium URL: https://aquarium.h2o.ai.

  2. Create a new Aquarium account.

  3. Check the registered email inbox and use the temporary password to login to Aquarium.

  4. Click Browse Labs.

  5. Click View Detail under Open Source MLI Workshop.

  6. Click Start Lab (this can take several minutes).

  7. Click on the Jupyter URL when it becomes available.

  8. Enter the token h2o.

  9. Click the patrick_hall_mli folder.

  10. Browse/run the Jupyter notebooks.

  11. Click End Lab when you are finished.

Virtualenv Installation

For avid Python users, creating a Python virtual environment is a convenient way to run these notebooks.

  1. Install Git.

  2. Clone this repository with the examples.
    $ git clone https://github.com/jphall663/interpretable_machine_learning_with_python.git

  3. Install Anaconda Python 5.1.0 from the Anaconda archives and add it to your system path.

  4. Change directories into the cloned repository.
    $ cd interpretable_machine_learning_with_python

  5. Create a Python 3.6 virtual environment.
    $ virtualenv -p /path/to/anaconda3/bin/python3.6 env_iml

  6. Activate the virtual environment.
    $ source env_iml/bin/activate

  7. Install the correct packages for the example notebooks.
    $ pip install -r requirements.txt

  8. Start Jupyter.
    $ jupyter notebook

Docker Installation

A Dockerfile is provided to build a docker container with all necessary packages and dependencies. This is a way to use these examples if you are on Mac OS X, *nix, or Windows 10. To do so:

  1. Install and start docker.

From a terminal:

  1. Create a directory for the Dockerfile.
    $ mkdir anaconda_py36_h2o_xgboost_graphviz_shap

  2. Fetch the Dockerfile.
    $ curl https://raw.githubusercontent.com/jphall663/interpretable_machine_learning_with_python/master/anaconda_py36_h2o_xgboost_graphviz_shap/Dockerfile > anaconda_py36_h2o_xgboost_graphviz_shap/Dockerfile

  3. Build a docker image from the Dockefile.
    docker build -t iml anaconda_py36_h2o_xgboost_graphviz_shap

  4. Start the docker image and the Jupyter notebook server.
    docker run -i -t -p 8888:8888 iml:latest /bin/bash -c "/opt/conda/bin/jupyter notebook --notebook-dir=/interpretable_machine_learning_with_python --allow-root --ip='*' --port=8888 --no-browser"

  5. Navigate to port 8888 on your machine, probably http://localhost:8888/.

Manual Installation

  1. Anaconda Python 5.1.0 from the Anaconda archives.
  2. Java.
  3. The latest stable h2o Python package.
  4. Git.
  5. XGBoost with Python bindings.
  6. GraphViz.
  7. Seaborn package.
  8. Shap package.

Anaconda Python, Java, Git, and GraphViz must be added to your system path.

From a terminal:

  1. Clone the repository with examples.
    $ git clone https://github.com/jphall663/interpretable_machine_learning_with_python.git

  2. $ cd interpretable_machine_learning_with_python

  3. Start the Jupyter notebook server.
    $ jupyter notebook

  4. Navigate to the port Jupyter directs you to on your machine, probably http://localhost:8888/.

Comments
  • Issue with Graphviz

    Issue with Graphviz

    Hey,

    Firstly, this is an amazing work of yours. I really admire what you have done. I am facing a small issue while displaying the png file. I cannot convert it from dot to png. Can you please help me with this?

    opened by mananshukla6 7
  • O'Reilly Safari videos not working

    O'Reilly Safari videos not working

    Hi @jphall663.

    Firstly, a huge shout out to you for this repo. I wanted to tell you that the videos that you linked are not working.

    For your reference: https://www.useloom.com/share/3c31bf9b1e4c44608f2b6940230cc11a

    It happens with the other three video links as well.

    opened by sayakpaul 2
  • Ph add errors

    Ph add errors

    New plan off attack on model debugging ...

    Sensitivity Analysis:

    • Random attacks (very important actually, even though they drive me insane)
    • PDP:
    • -- w/ out of range values
    • -- w/ missing values
    • -- w/ ICE at deciles of yhat
    • -- w/ residual of PD
    • -- ALWAYS w/ histogram so you can see where data supports predictions
    • Adversarial examples:
    • -- What can change high residual points to low residual points (and vice versa)
    • -- What can change incorrectly classified points to correctly classified point (and vice versa)
    • -- record non-nonsensical features and combinations that cause problems as non-robust
    • -- train model on wrongly classified points (high-noise training set) and look for important features; score on low-noise training data and conduct activation analysis to find non-robust features

    https://github.com/jphall663/interpretable_machine_learning_with_python/blob/ph_add_errors/debugging_sens_analysis_redux.ipynb

    Residual Analysis:

    • Always report more than one error metric
    • Analyze w.r.t. trusted benchmark model (similarity matrix?)
    • Plots of residual vs. prediction
    • Plots of residual vs. target and top-k important inputs, by level if categorical
    • Explanation of residual: DT surrogate (trained on right/wrong), LIME (trained on right/wrong), Shap, LOCO of residual (global and local)
    • Use explanation of residuals: DT surrogate, LIME, Shap, LOCO of residual (global and local) to automatically build tests/assertions/adversaries.
    • Plots of Shap/LOCO vs. residual

    https://github.com/jphall663/interpretable_machine_learning_with_python/blob/ph_add_errors/debugging_resid_analysis_redux.ipynb

    For each bullet above, try to discuss in terms of:

    • accuracy
    • disparate impact
    • privacy/security

    Potentially remove: https://github.com/jphall663/interpretable_machine_learning_with_python/blob/ph_add_errors/pdp_dt_surr_res_shap_loss.ipynb

    Self-Healing

    • Use discovered assertions in training to prevent error states
    • Use adversarial examples to find non-robust (non-nonsensical? use explanation to decide) features or interactions that are correlated to the target and remove, or prevent these interactions from being modeled.
    • Boosting of surrogate prediction model from the underlying oracle model to the point where they are as accurate as the oracle ("Dagger").
    • Different models on robust and non-robust features and give lower weight to non-robust features model in ensemble.
    opened by jphall663 1
  • Please post the slides from Youtube video https://www.youtube.com/watch?v=Q8rTrmqUQsU

    Please post the slides from Youtube video https://www.youtube.com/watch?v=Q8rTrmqUQsU

    I enjoyed your presentation I found on Youtube from a meet up in late April. You'd mentioned the pdf for the talk would be posted but from Youtube it is unable to be found. Could you please post here or there? Thanks in advance.

    opened by wnmills3 1
  • Add adverse impact ratio for dia.ipynb

    Add adverse impact ratio for dia.ipynb

    We need to add another metric:

    Adverse Impact: (tp + fp) / (tp + fp + tn + fn)

    Adverse Impact Disparity (Ratio): non-reference adverse impact / reference adverse impact

    Adverse Impact Parity: low_threshold < Adverse Impact Disparity < high_threshold

    opened by jphall663 0
  • Bump numpy from 1.16.0 to 1.22.0

    Bump numpy from 1.16.0 to 1.22.0

    Bumps numpy from 1.16.0 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Owner
ph_
Helping people manage AI, machine learning, and analytics risks @bnh-ai; GWU lecturer.
ph_
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

Vowpal Wabbit 8.1k Dec 30, 2022
CD) in machine learning projectsImplementing continuous integration & delivery (CI/CD) in machine learning projects

CML with cloud compute This repository contains a sample project using CML with Terraform (via the cml-runner function) to launch an AWS EC2 instance

Iterative 19 Oct 3, 2022
MIT-Machine Learning with Python–From Linear Models to Deep Learning

MIT-Machine Learning with Python–From Linear Models to Deep Learning | One of the 5 courses in MIT MicroMasters in Statistics & Data Science Welcome t

null 2 Aug 23, 2022
Microsoft contributing libraries, tools, recipes, sample codes and workshop contents for machine learning & deep learning.

Microsoft contributing libraries, tools, recipes, sample codes and workshop contents for machine learning & deep learning.

Microsoft 366 Jan 3, 2023
A data preprocessing package for time series data. Design for machine learning and deep learning.

A data preprocessing package for time series data. Design for machine learning and deep learning.

Allen Chiang 152 Jan 7, 2023
A mindmap summarising Machine Learning concepts, from Data Analysis to Deep Learning.

A mindmap summarising Machine Learning concepts, from Data Analysis to Deep Learning.

Daniel Formoso 5.7k Dec 30, 2022
A comprehensive repository containing 30+ notebooks on learning machine learning!

A comprehensive repository containing 30+ notebooks on learning machine learning!

Jean de Dieu Nyandwi 3.8k Jan 9, 2023
Implemented four supervised learning Machine Learning algorithms

Implemented four supervised learning Machine Learning algorithms from an algorithmic family called Classification and Regression Trees (CARTs), details see README_Report.

Teng (Elijah)  Xue 0 Jan 31, 2022
High performance, easy-to-use, and scalable machine learning (ML) package, including linear model (LR), factorization machines (FM), and field-aware factorization machines (FFM) for Python and CLI interface.

What is xLearn? xLearn is a high performance, easy-to-use, and scalable machine learning package that contains linear model (LR), factorization machin

Chao Ma 3k Jan 8, 2023
A library of extension and helper modules for Python's data analysis and machine learning libraries.

Mlxtend (machine learning extensions) is a Python library of useful tools for the day-to-day data science tasks. Sebastian Raschka 2014-2021 Links Doc

Sebastian Raschka 4.2k Dec 29, 2022
MLBox is a powerful Automated Machine Learning python library.

MLBox is a powerful Automated Machine Learning python library. It provides the following features: Fast reading and distributed data preprocessing/cle

Axel 1.4k Jan 6, 2023
Python package for stacking (machine learning technique)

vecstack Python package for stacking (stacked generalization) featuring lightweight functional API and fully compatible scikit-learn API Convenient wa

Igor Ivanov 671 Dec 25, 2022
A Python Package to Tackle the Curse of Imbalanced Datasets in Machine Learning

imbalanced-learn imbalanced-learn is a python package offering a number of re-sampling techniques commonly used in datasets showing strong between-cla

null 6.2k Jan 1, 2023
Extreme Learning Machine implementation in Python

Python-ELM v0.3 ---> ARCHIVED March 2021 <--- This is an implementation of the Extreme Learning Machine [1][2] in Python, based on scikit-learn. From

David C. Lambert 511 Dec 20, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 5, 2023
Python module for machine learning time series:

seglearn Seglearn is a python package for machine learning time series or sequences. It provides an integrated pipeline for segmentation, feature extr

David Burns 536 Dec 29, 2022
Uber Open Source 1.6k Dec 31, 2022
ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions

A library for debugging/inspecting machine learning classifiers and explaining their predictions

null 154 Dec 17, 2022
QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

null 152 Jan 2, 2023