New feature : hability to provide a custom executor (an object that behave like the mp.Pool, with a context manager behaviour and a map method.)

example using dask.distributed tool :

In [1]: import blackbox as bb
   ...: import numpy as np
   ...: from distributed import Client
   ...: 
   ...: c = Client("localhost:8786")
   ...: 
   ...: def fun(par):
   ...:     a, b = par
   ...:     return np.sqrt((a - 5) ** 2 + (b - 3) ** 2)
   ...: 
   ...: 
   ...: def main():
   ...:     bb.search(f=fun,  # given function
   ...:               box=[[-10., 10.], [-10., 10.]],  # range of values for each parameter
   ...:               n=4,  # number of function calls on initial stage (global search)
   ...:               m=4,  # number of function calls on subsequent stage (local search)
   ...:               batch=8,  # number of calls that will be evaluated in parallel
   ...:               resfile='output.csv',
   ...:               executor=c.get_executor)  # text file where results will be saved
   ...: 
   ...: 
   ...: if __name__ == '__main__':
   ...:     main()

That snippet launch the computation on the different dask workers whose can be on different computers on the network.

Thank you for providing this very useful toolbox!

When I try to use it for my optimization problem, it starts out quite well but it raises an error after having evaluated around half of the batches:

File "/Users/philipp/miniconda3/lib/python3.6/site-packages/black_box/blackbox.py", line 106, in search_min
  points[n+batch*i:n+batch*(i+1), -1] = list(e.map(f, list(map(cubetobox, points[n+batch*i:n+batch*(i+1), 0:-1]))))/fmax
TypeError: unsupported operand type(s) for /: 'NoneType' and 'float' 

It seems, that the map somehow returns a NoneType. I couldn't find the reason for it. Do you have any suggestions?

Unfortunately, I can not provide a MWE, since my cost-function is a very advanced black-box. The function is called with:

fit_par = bb.search_min(f=cost_function, domain=bounds_list, budget=50, batch=4, resfile = 'bb_output.csv')

Tried this on a Linux and MacOs Machine. I am working in a anaconda environment with Python 3.7.9 black-box 1.0.2

Upon further investigation I discorvered, that there is a nan appearing at the last position of all the sublists in the points list. Could this be the problem and how could this have happened?

For completeness, my domain is

[[1000000000.0, 30000000000.0],
 [1000000000.0, 30000000000.0],
 [2e+18, 5e+18],
 [-1e-07, 3e-07],
 [10000.0, 100000.0],
 [-5000.0, 5000.0],
 [10000.0, 100000.0],
 [-5000.0, 5000.0]]

Hello,

I found your module pretty useful so I tried to package it under a setup.py so it can be installed using python setup.py install

I basically named the package as blackboxfunctions. If you don't like the name you can just change it or I make another pull request with another name that you like. I could not name it blackbox because a package like that already exists so there would be conflicts in case you want to upload this to PyPI

I also moved blackbox.py under blackboxfunctions directory and created an init.py that basically imports everything so it can be used as

import blackboxfunctions as bb

Also I created a .gitignore

You can check the information of the module in setup.py. I specified Paul Knysh as the author and used the email I found on his github page for the author_email field (not sure if you like this).

Lastly I changed the way blackbox prints to the output by using python's logging module. So for the error messages I used logger.error, for warnings logger.warning and for info messages logger.info. By doing this the user can actually control the output levels of the blackbox module by using logging.

I assumed that you would like to keep functionality for python 2.7 even though it is being deprecated, so I made sure that it works for both versions. I tested the setup.py and the module itself against python 2.7 and python 3.8 and they work as expected.

Also I have a branch in my blackbox repository named logging (https://github.com/tchar/blackbox/tree/logging) with the only change to be the usage of python's logging module without rearranging the structure, no setup.py and no renaming. So if you don't want the changes made for setup.py let me know so I make a pull request just for the logging change.

Please let me know what you think of the changes

Cheers, tchar

I'm looking for an adaptation of your method to allow joint constraints on parameters, not individually (i.e. range of value for each params). For instance: param1 + param2 + param3 <1
0<param1<1
0<param2<1
0<param3<1 I can constraint range values of param1, param2 and param3 in [0, 1] but not the sum ?

Thank you

For a high dimensional problem, one naive approach is to calculate all the partial derivative in parallel then apply SGD or other algorithm. Have you compare this naive approach with your method, say in term of numbers of function calls and convergence rate.

Hello! I thought you might be interested in a direct way of making points. It uses low discrepancy quasirandom sequence (with seed) to fill hypercube faster than minimizing spread function and with better spread (24k vs 27k with n=300, d=22). From http://extremelearning.com.au/unreasonable-effectiveness-of-quasirandom-sequences/ Regards, Andrii

Hello, thank you for publishing your project! I'm no optimization expert, but I was interested in learning about your parallel optimization procedure.

I reformatted the code into a more conventional python style (see the scipy.optimize module for example). This will make the code more readable to the average user. I followed the numpy documentation guide for doctrings (https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt).

I also added a script (example.py) with an example from your arXiv note.

Best regards, Drew

Hello,

Do you have a comparison with scipy's minimize function (https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.optimize.minimize.html#scipy.optimize.minimize), maybe even using the different methods?

I have created a simulation (https://github.com/pedvide/simetuc) that takes from 1s to maybe 1000s to solve. optimization of the few (1-5) free parameters is painfully slow for the longer simulations, so I'm interested in any improvement!

Hello Paul,

Thank you for this nice piece of code :+1: Is it possible to feed the algorithm with an initial guess ? Or can you think of some clues to implement this ?

I would like to use Blackbox as a part of my reactive-transport modeling tool for calibration. How would you like me to cite you in my Github project?

It could be a good idea to have a pool argument which take an object with a map method. That way, it will be easy to use other multiprocessing tools like ipyparallel, dask.distributed or pathos.

See abcpmc for a similar implementation.

The sample code from the readme fails on my Python 3.10.5 install.

result = bb.minimize(f=fun, # given function
    domain=[[-5, 5], [-5, 5]], # ranges of each parameter
    budget=20, # total number of function calls available
    batch=4 # number of calls that will be evaluated in parallel
)
print(result["best_x"])
print(result["best_f"])

Leads to:

INFO     evaluating batch 1/5 (samples 1..4/20) 10-06 10:54:00
INFO     evaluating batch 2/5 (samples 5..8/20) 10-06 10:54:04
INFO     evaluating batch 3/5 (samples 9..12/20) 10-06 10:54:09
Traceback (most recent call last):
  File "Optimierung_Logistikkosten_BlackBox.py", line 273, in <module>
    main()
  File "Optimierung_Logistikkosten_BlackBox.py", line 208, in main
    result = bb.minimize(f=fun, # given function
  File "\blackbox\blackbox.py", line 86, in minimize
    v1 = compute_volume_unit_ball(d)
  File "\blackbox\blackbox.py", line 165, in compute_volume_unit_ball
    v1 = np.pi ** (d / 2) / np.math.factorial(d / 2)
TypeError: 'float' object cannot be interpreted as an integer

Note that I have redacted the path information.

https://bugs.python.org/issue37315 Accepting floats for math.factorial was removed. since d is either even or manipulated to be even, it should be safe to convert to an integer. https://stackoverflow.com/questions/21753841/factorial-in-numpy-and-scipy math.factorial and np.math.factorial is the same thing

Hello Paul.

Is it possible to pass function arguments to the solver? For example, my function is defined as:

def func(x,*parms):

And then with the scipy minimize I can pass the parameters with the args= argument:

minimize(func, Initial_Guesses, method = 'SLSQP', bounds = Bounds, args=Parms )

I looked at the code but did not see a way to pass this argument. Thanks

Is there a way to limit parameters to integers only? I'm currently rounding the parameter in the objective function, but this leads to the same value being evaluated multiple times.

Using a distributed job submission platform to submit the function evaluation calls to a cluster. It seemed that the number of evaluations run in parallel was lower than what I set it to with the "batch" parameter. Reading through the Python doc for multiprocessing.Pool, the default for the processesargument is the number of cores on the local machine. Setting processes=batch solved the problem, allowing the number of evaluations set to run in parallel.