- The template of Bayesian optimization Hyperband (BOHB)
python main.py
in 2021-10-22
- tf keras version (tunecallback) completed.
- tf gradienttape version (train_iteration) completed.
- pytorch version completed.
pip install ray[tune] hpbandster ConfigSpace
- The epoch of keras or torch is terminated based on max_t value.
Running example
+-----------------------+----------+-------+--------------+-----------+-----------+--------------+
| Trial name | status | loc | activation | neuron1 | neuron2 | optimizers |
|-----------------------+----------+-------+--------------+-----------+-----------+--------------|
| objective_dd8c7_00000 | PENDING | | tanh | 43 | 50 | adam |
| objective_dd8c7_00001 | PENDING | | relu | 63 | 45 | adam |
| objective_dd8c7_00002 | PENDING | | tanh | 63 | 52 | rmsprop |
+-----------------------+----------+-------+--------------+-----------+-----------+--------------+
- It can be stopped according to the value of t in the stop item of tune.run.
For example,
stop={
"mean_accuracy": 0.99, # Terminate if accuracy is greater than 0.99
"training_iteration": 1
},
In this case, no matter how long max_t is, when all trials are running (just training iteration one), it is considered as ends.
Number of trials: 3/3 (3 TERMINATED)
+-----------------------+------------+-------+--------------+-----------+-----------+--------------+----------+--------+------------------+
| Trial name | status | loc | activation | neuron1 | neuron2 | optimizers | acc | iter | total time (s) |
|-----------------------+------------+-------+--------------+-----------+-----------+--------------+----------+--------+------------------|
| objective_dd8c7_00000 | TERMINATED | | tanh | 43 | 50 | adam | 0.938278 | 1 | 5.3518 |
| objective_dd8c7_00001 | TERMINATED | | relu | 63 | 45 | adam | 0.942333 | 1 | 5.15036 |
| objective_dd8c7_00002 | TERMINATED | | tanh | 63 | 52 | rmsprop | 0.94 | 1 | 4.28608 |
+-----------------------+------------+-------+--------------+-----------+-----------+--------------+----------+--------+------------------+
- Records are accumulated in the specified path during execution, which can be checked with the tensorboard command.
tensorboard --logdir bohb_results/
