cuML - GPU Machine Learning Algorithms
cuML - GPU Machine Learning Algorithms
cuML is a suite of libraries that implement machine learning algorithms and mathematical primitives functions that share compatible APIs with other RAPIDS projects.
cuML enables data scientists, researchers, and software engineers to run traditional tabular ML tasks on GPUs without going into the details of CUDA programming. In most cases, cuML's Python API matches the API from scikit-learn.
For large datasets, these GPU-based implementations can complete 10-50x faster than their CPU equivalents. For details on performance, see the cuML Benchmarks Notebook.
As an example, the following Python snippet loads input and computes DBSCAN clusters, all on GPU, using cuDF:
import cudf
from cuml.cluster import DBSCAN
# Create and populate a GPU DataFrame
gdf_float = cudf.DataFrame()
gdf_float['0'] = [1.0, 2.0, 5.0]
gdf_float['1'] = [4.0, 2.0, 1.0]
gdf_float['2'] = [4.0, 2.0, 1.0]
# Setup and fit clusters
dbscan_float = DBSCAN(eps=1.0, min_samples=1)
dbscan_float.fit(gdf_float)
print(dbscan_float.labels_)
Output:
0 0
1 1
2 2
dtype: int32
cuML also features multi-GPU and multi-node-multi-GPU operation, using Dask, for a growing list of algorithms. The following Python snippet reads input from a CSV file and performs a NearestNeighbors query across a cluster of Dask workers, using multiple GPUs on a single node:
Initialize a LocalCUDACluster configured with UCX for fast transport of CUDA arrays
# Initialize UCX for high-speed transport of CUDA arrays
from dask_cuda import LocalCUDACluster
# Create a Dask single-node CUDA cluster w/ one worker per device
cluster = LocalCUDACluster(protocol="ucx",
enable_tcp_over_ucx=True,
enable_nvlink=True,
enable_infiniband=False)
Load data and perform k-Nearest Neighbors search. cuml.dask estimators also support Dask.Array as input:
from dask.distributed import Client
client = Client(cluster)
# Read CSV file in parallel across workers
import dask_cudf
df = dask_cudf.read_csv("/path/to/csv")
# Fit a NearestNeighbors model and query it
from cuml.dask.neighbors import NearestNeighbors
nn = NearestNeighbors(n_neighbors = 10, client=client)
nn.fit(df)
neighbors = nn.kneighbors(df)
For additional examples, browse our complete API documentation, or check out our example walkthrough notebooks. Finally, you can find complete end-to-end examples in the notebooks-contrib repo.
Supported Algorithms
| Category | Algorithm | Notes |
|---|---|---|
| Clustering | Density-Based Spatial Clustering of Applications with Noise (DBSCAN) | Multi-node multi-GPU via Dask |
| K-Means | Multi-node multi-GPU via Dask | |
| Dimensionality Reduction | Principal Components Analysis (PCA) | Multi-node multi-GPU via Dask |
| Incremental PCA | ||
| Truncated Singular Value Decomposition (tSVD) | Multi-node multi-GPU via Dask | |
| Uniform Manifold Approximation and Projection (UMAP) | Multi-node multi-GPU Inference via Dask | |
| Random Projection | ||
| t-Distributed Stochastic Neighbor Embedding (TSNE) | ||
| Linear Models for Regression or Classification | Linear Regression (OLS) | Multi-node multi-GPU via Dask |
| Linear Regression with Lasso or Ridge Regularization | Multi-node multi-GPU via Dask | |
| ElasticNet Regression | ||
| LARS Regression | (experimental) | |
| Logistic Regression | Multi-node multi-GPU via Dask-GLM demo | |
| Naive Bayes | Multi-node multi-GPU via Dask | |
| Stochastic Gradient Descent (SGD), Coordinate Descent (CD), and Quasi-Newton (QN) (including L-BFGS and OWL-QN) solvers for linear models | ||
| Nonlinear Models for Regression or Classification | Random Forest (RF) Classification | Experimental multi-node multi-GPU via Dask |
| Random Forest (RF) Regression | Experimental multi-node multi-GPU via Dask | |
| Inference for decision tree-based models | Forest Inference Library (FIL) | |
| K-Nearest Neighbors (KNN) Classification | Multi-node multi-GPU via Dask+UCX, uses Faiss for Nearest Neighbors Query. | |
| K-Nearest Neighbors (KNN) Regression | Multi-node multi-GPU via Dask+UCX, uses Faiss for Nearest Neighbors Query. | |
| Support Vector Machine Classifier (SVC) | ||
| Epsilon-Support Vector Regression (SVR) | ||
| Time Series | Holt-Winters Exponential Smoothing | |
| Auto-regressive Integrated Moving Average (ARIMA) | Supports seasonality (SARIMA) | |
| Model Explanation | SHAP Kernel Explainer | Based on SHAP (experimental) |
| SHAP Permutation Explainer | Based on SHAP (experimental) | |
| Other | K-Nearest Neighbors (KNN) Search | Multi-node multi-GPU via Dask+UCX, uses Faiss for Nearest Neighbors Query. |
Installation
See the RAPIDS Release Selector for the command line to install either nightly or official release cuML packages via Conda or Docker.
Build/Install from Source
See the build guide.
Contributing
Please see our guide for contributing to cuML.
References
The RAPIDS team has a number of blogs with deeper technical dives and examples. You can find them here on Medium.
For additional details on the technologies behind cuML, as well as a broader overview of the Python Machine Learning landscape, see Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence (2020) by Sebastian Raschka, Joshua Patterson, and Corey Nolet.
Please consider citing this when using cuML in a project. You can use the citation BibTeX:
@article{raschka2020machine,
title={Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligence},
author={Raschka, Sebastian and Patterson, Joshua and Nolet, Corey},
journal={arXiv preprint arXiv:2002.04803},
year={2020}
}
Contact
Find out more details on the RAPIDS site
Open GPU Data Science
The RAPIDS suite of open source software libraries aim to enable execution of end-to-end data science and analytics pipelines entirely on GPUs. It relies on NVIDIA® CUDA® primitives for low-level compute optimization, but exposing that GPU parallelism and high-bandwidth memory speed through user-friendly Python interfaces.
![[QST] Best practices to work with big datasets in RandomForestRegressor](https://avatars.githubusercontent.com/u/58592703?v=4)
![[QST] Best practices to achieve greater max_depth and n_trees parameters in RandomForestRegressor](https://avatars.githubusercontent.com/u/21284872?v=4)
![[BUG] RMM-only context destroyed error with Random Forest in loop](https://avatars.githubusercontent.com/u/8457388?v=4)
![[REVIEW] Symbolic Regression/Classification C/C++](https://avatars.githubusercontent.com/u/39610523?v=4)
![[BUG] Bootstrapping causes accuracy drop in cuML RF](https://avatars.githubusercontent.com/u/2532981?v=4)
![[REVIEW] Refactor src prims for header name rules](https://avatars.githubusercontent.com/u/8596659?v=4)
![[WIP] LOBPCG Solver](https://avatars.githubusercontent.com/u/23023424?v=4)
![[BUG] make_regression() much slower after certain dataset size.](https://avatars.githubusercontent.com/u/26169771?v=4)
![[REVIEW] Enable clang format](https://avatars.githubusercontent.com/u/532947?v=4)
![[REVIEW] Proposal for](https://avatars.githubusercontent.com/u/3633935?v=4)
![[BUG] lasso interop test failure in CI](https://avatars.githubusercontent.com/u/1116796?v=4)
![[FEA] Add support for more TfIdf parameters](https://avatars.githubusercontent.com/u/9810050?v=4)
646 Dec 30, 2022
1.8k Jan 2, 2023
4.2k Jan 8, 2023
6.6k Jan 5, 2023
4k Dec 29, 2022
212 Jan 4, 2023
95 Jan 1, 2023
657 Dec 19, 2022
6.1k Jan 5, 2023
21.8k Jan 9, 2023
8.1k Jan 6, 2023
8.9k Dec 30, 2022
84 Nov 25, 2022
81 Nov 26, 2022