"To deal with hyper-planes in a 14 dimensional space, visualize a 3D space and say 'fourteen' very loudly. Everyone does it." - Geoff Hinton
Overview
HyperTools is designed to facilitate dimensionality reduction-based visual explorations of high-dimensional data. The basic pipeline is to feed in a high-dimensional dataset (or a series of high-dimensional datasets) and, in a single function call, reduce the dimensionality of the dataset(s) and create a plot. The package is built atop many familiar friends, including matplotlib, scikit-learn and seaborn. Our package was recently featured on Kaggle's No Free Hunch blog. For a general overview, you may find this talk useful (given as part of the MIND Summer School at Dartmouth).
Try it!
Click the badge to launch a binder instance with example uses:
or
Check the repo of Jupyter notebooks from the HyperTools paper.
Installation
To install the latest stable version run:
pip install hypertools
To install the latest unstable version directly from GitHub, run:
pip install -U git+https://github.com/ContextLab/hypertools.git
Or alternatively, clone the repository to your local machine:
git clone https://github.com/ContextLab/hypertools.git
Then, navigate to the folder and type:
pip install -e .
(These instructions assume that you have pip installed on your system)
NOTE: If you have been using the development version of 0.5.0, please clear your data cache (/Users/yourusername/hypertools_data).
Requirements
- python 2.7, 3.5+
- PPCA>=0.0.2
- scikit-learn>=0.18.1
- pandas>=0.18.0
- seaborn>=0.8.1
- matplotlib>=1.5.1
- scipy>=0.17.1
- numpy>=1.10.4
- future
- requests
- deepdish
- pytest (for development)
- ffmpeg (for saving animations)
If installing from github (instead of pip), you must also install the requirements: pip install -r requirements.txt
Troubleshooting
If you encounter an error related to installing deepdish (hdf5) on a MacOS system, try installing hdf5 directly using homebrew:
$ brew tap homebrew/science
$ brew install hdf5
and then re-start the installation.
Documentation
Check out our readthedocs page for further documentation, complete API details, and additional examples.
Citing
We wrote a short JMLR paper about HyperTools, which you can read here, or you can check out a (longer) preprint here. We also have a repository with example notebooks from the paper here.
Please cite as:
Heusser AC, Ziman K, Owen LLW, Manning JR (2018) HyperTools: A Python toolbox for gaining geometric insights into high-dimensional data. Journal of Machine Learning Research, 18(152): 1--6.
Here is a bibtex formatted reference:
@ARTICLE {,
author = {Andrew C. Heusser and Kirsten Ziman and Lucy L. W. Owen and Jeremy R. Manning},
title = {HyperTools: a Python Toolbox for Gaining Geometric Insights into High-Dimensional Data},
journal = {Journal of Machine Learning Research},
year = {2018},
volume = {18},
number = {152},
pages = {1-6},
url = {http://jmlr.org/papers/v18/17-434.html}
}
Contributing
If you'd like to contribute, please first read our Code of Conduct.
For specific information on how to contribute to the project, please see our Contributing page.
Testing
To test HyperTools, install pytest (pip install pytest) and run pytest in the HyperTools folder
Examples
See here for more examples.
Plot
import hypertools as hyp
hyp.plot(list_of_arrays, '.', group=list_of_labels)
Align
import hypertools as hyp
hyp.plot(list_of_arrays, align='hyper')
BEFORE
AFTER
Cluster
import hypertools as hyp
hyp.plot(array, '.', n_clusters=10)
Describe
import hypertools as hyp
hyp.tools.describe(list_of_arrays, reduce='PCA', max_dims=14)
138 Dec 6, 2022
2.4k Jan 4, 2023
36 Sep 28, 2022
0 Jul 17, 2021
723 Jan 7, 2023
98 Dec 29, 2022
3 Jul 9, 2022
3 Jul 15, 2022
1k Jan 1, 2023
26 Dec 17, 2022
3 Jan 19, 2022
502 Dec 28, 2022
11 Dec 5, 2022
91 Dec 29, 2022
697 Jan 6, 2023
2.4k Jan 7, 2023