easyopt is a super simple yet super powerful optuna-based Hyperparameters Optimization Framework that requires no coding.

Overview

easyopt

easyopt is a super simple yet super powerful optuna-based Hyperparameters Optimization Framework that requires no coding.

Features

  • YAML Configuration
  • Distributed Parallel Optimization
  • Experiments Monitoring and Crash Recovering
  • Experiments Replicas
  • Real Time Pruning
  • A wide variety of sampling strategies
    • Tree-structured Parzen Estimator
    • CMA-ES
    • Grid Search
    • Random Search
  • A wide variety of pruning strategies
    • Asynchronous Successive Halving Pruning
    • Hyperband Pruning
    • Median Pruning
    • Threshold Pruning
  • A wide variety of DBMSs
    • Redis
    • SQLite
    • PostgreSQL
    • MySQL
    • Oracle
    • And many more

Installation

To install easyopt just type:

pip install easyopt

Example

easyopt expects that hyperparameters are passed using the command line arguments.

For example this problem has two hyperparameters x and y

import argparse

parser = argparse.ArgumentParser()

parser.add_argument("--x", type=float, required=True)
parser.add_argument("--y", type=float, required=True)

args = parser.parse_args()

def objective(x, y):
    return x**2 + y**2

F = objective(args.x ,args.y)

To integrate easyopt you just have to

  • import easyopt
  • Add easyopt.objective(...) to report the experiment objective function value

The above code becomes:

import argparse
import easyopt

parser = argparse.ArgumentParser()

parser.add_argument("--x", type=float, required=True)
parser.add_argument("--y", type=float, required=True)

args = parser.parse_args()

def objective(x, y):
    return x**2 + y**2

F = objective(args.x ,args.y)
easyopt.objective(F)

Next you have to create the easyopt.yml to define the problem search space, sampler, pruner, storage, etc.

command: python problem.py {args}
storage: sqlite:////tmp/easyopt-toy-problem.db
sampler: TPESampler
parameters:
  x:
    distribution: uniform
    low: -10
    high: 10
  y:
    distribution: uniform
    low: -10
    high: 10

You can find the compete list of distributions here (all the suggest_* functions)

Finally you have to create a study

easyopt create test-study

And run as many agents as you want

easyopt agent test-study

After a while the hyperparameter optimization will finish

Trial 0 finished with value: 90.0401543850028 and parameters: {'x': 5.552902529323713, 'y': 7.694506344453366}. Best is trial 0 with value: 90.0401543850028.
Trial 1 finished with value: 53.38635524683359 and parameters: {'x': 0.26609756303111, 'y': 7.301749607716118}. Best is trial 1 with value: 53.38635524683359.
Trial 2 finished with value: 64.41207387363161 and parameters: {'x': 7.706366704967074, 'y': 2.2414250115064167}. Best is trial 1 with value: 53.38635524683359.
...
...
Trial 53 finished with value: 0.5326245807950265 and parameters: {'x': -0.26584110075742917, 'y': 0.6796713102251005}. Best is trial 35 with value: 0.11134607529340049.
Trial 54 finished with value: 8.570230212116037 and parameters: {'x': 2.8425893061307295, 'y': 0.6999401751487438}. Best is trial 35 with value: 0.11134607529340049.
Trial 55 finished with value: 96.69479467451664 and parameters: {'x': -0.3606041968175481, 'y': -9.826736960342137}. Best is trial 35 with value: 0.11134607529340049.

YAML Structure

The YAML configuration file is structured as follows

command: 
storage: 
   
sampler: 
   
pruner: 
   
direction: 
   
replicas: 
   
parameters:
  parameter-1:
    distribution: 
   
    
   : 
   
    
   : 
   
    ...
  ...
  • command: the command to execute to run the experiment.
    • {args} will be expanded to --parameter-1=value-1 --parameter-2=value-2
    • {name} will be expanded to the study name
  • storage: the storage to use for the study. A full list of storages is available here
  • sampler: the sampler to use. The full list of samplers is available here
  • pruner: the pruner to use. The full list of pruners is available here
  • direction: can be minimize or maximize (default: minimize)
  • replicas: the number of replicas to run for the same experiment (the experiment result is the average). (default: 1)
  • parameters: the parameters to optimize
    • for each parameter have to specify
      • distribution the distribution to use. The full list of distributions is available here (all the suggest_* functions)
      • arg: value
        • Arguments of the distribution. The arguments documentation is available here

CLI Interface

easyopt offer two CLI commands:

  • create to create a study using the easyopt.yml file or the one specified with --config
  • agent to run the agent for

LIB Interface

When importing easyopt you can use three functions:

  • easyopt.objective(value) to report the final objective function value of the experiment
  • easyopt.report(value) to report the current objective function value of the experiment (used by the pruner)
  • easyopt.should_prune() it returns True if the pruner thinks that the run should be pruned

Examples

You can find some examples here

Contributions and license

The code is released as Free Software under the GNU/GPLv3 license. Copying, adapting and republishing it is not only allowed but also encouraged.

For any further question feel free to reach me at [email protected] or on Telegram @galatolo

Owner
Federico Galatolo
PhD Student @ University of Pisa
Federico Galatolo
The no-nonsense, minimalist REST and app backend framework for Python developers, with a focus on reliability, correctness, and performance at scale.

The Falcon Web Framework Falcon is a reliable, high-performance Python web framework for building large-scale app backends and microservices. It encou

Falconry 8.6k Oct 21, 2021
Serverless Python

Zappa - Serverless Python About Installation and Configuration Running the Initial Setup / Settings Basic Usage Initial Deployments Updates Rollback S

Rich Jones 11.8k Oct 22, 2021
Serverless Python

Zappa - Serverless Python About Installation and Configuration Running the Initial Setup / Settings Basic Usage Initial Deployments Updates Rollback S

Rich Jones 11.7k Feb 17, 2021
The Modern And Developer Centric Python Web Framework. Be sure to read the documentation and join the Slack channel questions: http://slack.masoniteproject.com

NOTE: Masonite 2.3 is no longer compatible with the masonite-cli tool. Please uninstall that by running pip uninstall masonite-cli. If you do not unin

Masonite 1.7k Oct 23, 2021
FastAPI framework, high performance, easy to learn, fast to code, ready for production

FastAPI framework, high performance, easy to learn, fast to code, ready for production Documentation: https://fastapi.tiangolo.com Source Code: https:

Sebastián Ramírez 37.2k Oct 23, 2021
Django Ninja - Fast Django REST Framework

Django Ninja is a web framework for building APIs with Django and Python 3.6+ type hints.

Vitaliy Kucheryaviy 1.6k Oct 23, 2021
A comprehensive reference for all topics related to building and maintaining microservices

This pandect (πανδέκτης is Ancient Greek for encyclopedia) was created to help you find and understand almost anything related to Microservices that i

Ivan Bilan 45 Sep 30, 2021
Flask-Potion is a RESTful API framework for Flask and SQLAlchemy, Peewee or MongoEngine

Flask-Potion Description Flask-Potion is a powerful Flask extension for building RESTful JSON APIs. Potion features include validation, model resource

DTU Biosustain 481 Oct 6, 2021
Flask-Potion is a RESTful API framework for Flask and SQLAlchemy, Peewee or MongoEngine

Flask-Potion Description Flask-Potion is a powerful Flask extension for building RESTful JSON APIs. Potion features include validation, model resource

DTU Biosustain 484 Feb 3, 2021
Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.

Read Latest Documentation - Browse GitHub Code Repository hug aims to make developing Python driven APIs as simple as possible, but no simpler. As a r

Hug API Framework 6.6k Oct 18, 2021
The web framework for inventors

Emmett is a full-stack Python web framework designed with simplicity in mind. The aim of Emmett is to be clearly understandable, easy to be learned an

Emmett 716 Oct 17, 2021
web.py is a web framework for python that is as simple as it is powerful.

web.py is a web framework for Python that is as simple as it is powerful. Visit http://webpy.org/ for more information. The latest stable release 0.62

null 5.6k Oct 15, 2021
WebSocket and WAMP in Python for Twisted and asyncio

Autobahn|Python WebSocket & WAMP for Python on Twisted and asyncio. Quick Links: Source Code - Documentation - WebSocket Examples - WAMP Examples Comm

Crossbar.io 2.3k Oct 12, 2021
A familiar HTTP Service Framework for Python.

Responder: a familiar HTTP Service Framework for Python Powered by Starlette. That async declaration is optional. View documentation. This gets you a

Taoufik 3.6k Oct 15, 2021
Appier is an object-oriented Python web framework built for super fast app development.

Joyful Python Web App development Appier is an object-oriented Python web framework built for super fast app development. It's as lightweight as possi

Hive Solutions 116 Oct 2, 2021
bottle.py is a fast and simple micro-framework for python web-applications.

Bottle: Python Web Framework Bottle is a fast, simple and lightweight WSGI micro web-framework for Python. It is distributed as a single file module a

Bottle Micro Web Framework 7.4k Oct 23, 2021
The little ASGI framework that shines. ?

✨ The little ASGI framework that shines. ✨ Documentation: https://www.starlette.io/ Community: https://discuss.encode.io/c/starlette Starlette Starlet

Encode 6.2k Oct 24, 2021
PipeLayer is a lightweight Python pipeline framework

PipeLayer is a lightweight Python pipeline framework. Define a series of steps, and chain them together to create modular applications

greaterthan 60 Sep 3, 2021
Phoenix LiveView but for Django

Reactor, a LiveView library for Django Reactor enables you to do something similar to Phoenix framework LiveView using Django Channels. What's in the

Eddy Ernesto del Valle Pino 349 Oct 13, 2021