Hyperopt: Distributed Hyperparameter Optimization
Hyperopt is a Python library for serial and parallel optimization over awkward search spaces, which may include real-valued, discrete, and conditional dimensions.
Getting started
Install hyperopt from PyPI
$ pip install hyperopt
to run your first example
# define an objective function
def objective(args):
case, val = args
if case == 'case 1':
return val
else:
return val ** 2
# define a search space
from hyperopt import hp
space = hp.choice('a',
[
('case 1', 1 + hp.lognormal('c1', 0, 1)),
('case 2', hp.uniform('c2', -10, 10))
])
# minimize the objective over the space
from hyperopt import fmin, tpe, space_eval
best = fmin(objective, space, algo=tpe.suggest, max_evals=100)
print(best)
# -> {'a': 1, 'c2': 0.01420615366247227}
print(space_eval(space, best))
# -> ('case 2', 0.01420615366247227}
Contributing
Setup (based on this)
If you're a developer and wish to contribute, please follow these steps:
-
Create an account on GitHub if you do not already have one.
-
Fork the project repository: click on the ‘Fork’ button near the top of the page. This creates a copy of the code under your account on the GitHub user account. For more details on how to fork a repository see this guide.
-
Clone your fork of the hyperopt repo from your GitHub account to your local disk:
$ git clone https://github.com/<github username>/hyperopt.git $ cd hyperopt
Setup a python 3.x environment for dependencies
-
Create environment with:
$ python3 -m venv my_envor$ python -m venv my_envor with conda:
$ conda create -n my_env python=3 -
Activate the environment:
$ source my_env/bin/activate
or with conda:
$ conda activate my_env -
Install dependencies for extras (you'll need these to run pytest): Linux/UNIX:
$ pip install -e '.[MongoTrials, SparkTrials, ATPE, dev]'or Windows:
pip install -e .[MongoTrials] pip install -e .[SparkTrials] pip install -e .[ATPE] pip install -e .[dev]
-
Add the upstream remote. This saves a reference to the main hyperopt repository, which you can use to keep your repository synchronized with the latest changes:
$ git remote add upstream https://github.com/hyperopt/hyperopt.gitYou should now have a working installation of hyperopt, and your git repository properly configured. The next steps now describe the process of modifying code and submitting a PR:
-
Synchronize your master branch with the upstream master branch:
$ git checkout master $ git pull upstream master
-
Create a feature branch to hold your development changes:
$ git checkout -b my_featureand start making changes. Always use a feature branch. It’s good practice to never work on the master branch!
Formatting
-
We recommend to use Black to format your code before submitting a PR which is installed automatically in step 4.
-
Then, once you commit ensure that git hooks are activated (Pycharm for example has the option to omit them). This will run black automatically on all files you modified, failing if there are any files requiring to be blacked. In case black does not run execute the following:
$ black {source_file_or_directory} -
Develop the feature on your feature branch on your computer, using Git to do the version control. When you’re done editing, add changed files using git add and then git commit:
$ git add modified_files $ git commit -m "my first hyperopt commit"
Running tests
-
The tests for this project use PyTest and can be run by calling
pytest. -
Record your changes in Git, then push the changes to your GitHub account with:
$ git push -u origin my_feature
Note that dev dependencies require python 3.6+.
Algorithms
Currently three algorithms are implemented in hyperopt:
Hyperopt has been designed to accommodate Bayesian optimization algorithms based on Gaussian processes and regression trees, but these are not currently implemented.
All algorithms can be parallelized in two ways, using:
Documentation
Hyperopt documentation can be found here, but is partly still hosted on the wiki. Here are some quick links to the most relevant pages:
Related Projects
Examples
See projects using hyperopt on the wiki.
Announcements mailing list
Discussion mailing list
Cite
If you use this software for research, please cite the paper (http://proceedings.mlr.press/v28/bergstra13.pdf) as follows:
Bergstra, J., Yamins, D., Cox, D. D. (2013) Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. TProc. of the 30th International Conference on Machine Learning (ICML 2013), June 2013, pp. I-115 to I-23.
Thanks
This project has received support from
- National Science Foundation (IIS-0963668),
- Banting Postdoctoral Fellowship program,
- National Science and Engineering Research Council of Canada (NSERC),
- D-Wave Systems, Inc.
![[Feature] Support SparkTrials for SparkML models](https://avatars.githubusercontent.com/u/82044803?v=4)
555 Dec 24, 2022
214 Jan 8, 2023
2.1k Jan 3, 2023
398 Nov 9, 2022
1.6k Dec 15, 2022
20 Dec 9, 2021
7 Sep 6, 2022
136 Jan 8, 2023
8 Mar 31, 2022
8.4k Jan 1, 2023
163 Dec 26, 2022
1.3k Nov 17, 2022
6 Dec 14, 2022
3.7k Jan 3, 2023
1.4k Dec 25, 2022
25 Jul 27, 2022
21 Oct 5, 2022
69 Dec 26, 2022
38 Dec 12, 2022