Hello,

I would like to check if it is something to be expected that if I use PESM, it can take many hours to do 50 suggestions, while PES can do it in ca 10 minutes? or maybe it's one of the dependencies causing it? I had some trouble setting up PyGMO and NLOpt on Ubuntu 16.04 and downgraded my system to Ubuntu 14.04 on which everything seemed to compile fine.

Thanks! Martin

Dear all I'm a new user and trying to install Spearmint package for Bayesian optimization and I have problem using the software. My problem is with the third step that is mentioned in the site:

STEP 3: Running spearmint

1. Start up a MongoDB daemon instance: mongod --fork --logpath <path/to/logfile> --dbpath <path/to/dbfolder>
2. Run spearmint: python main.py </path/to/experiment/directory>)

I don't know where is the logpath and dbpath for mongod..... Would you please help me to install the package and run the branin example!

Thank you in advance, Best wishes, Mohammad

Dear all I've installed the spearmint package and mongoDB on Linux. My problem is in the final step, where I want to run main.py for testing branin.py, first I go to this address ("Desktop/Spearmint-master/examples/distributed/") and execute the branin.py Then, in the same address I execute the command " mongod --fork --logpath --dbpath" everything is OK to this step! for running the main.py, first I go back to this address("Desktop/Spearmint-master/Spearmint") and when I execute the command I see the following error!

Traceback (most recent call last): File "main.py", line 494, in main() File "main.py", line 249, in main options, expt_dir = get_options() File "main.py", line 216, in get_options expt_dir = os.path.realpath(os.path.expanduser(args[0])) IndexError: list index out of range

I'm wondering where we have to define the branin parameters in the main.py ??? would you please help me to solve the error? Thank you in advance, Mohammad

Dear all I'm trying to test the branin.py on the cluster (On my laptop every thing is OK), and I have the following error when I try to start mongo using command ; "mongod --fork --logpath --dbpath".

ERROR: child process failed, exited with error number 100

I have to mention that, when I type "mongo" on the cluster it says:

MongoDB shell version: 2.4.9 connecting to: test Server has startup warnings: Mon May 11 13:09:59.173 [initandlisten] Mon May 11 13:09:59.173 [initandlisten] ** WARNING: You are running on a NUMA machine. Mon May 11 13:09:59.173 [initandlisten] ** We suggest launching mongod like this to avoid performance problems: Mon May 11 13:09:59.173 [initandlisten] ** numactl --interleave=all mongod [other options] Mon May 11 13:09:59.173 [initandlisten]

So, I tried starting mongo using command; "numactl --interleave=all mongod --fork --logpath --dbpath", but didn't see any difference in the output!

Thanks, Mohammad

Hello,

I have been using the Spearmint tool from another codebase (JasperSnoek/spearmint), and I usually use the spearmint-lite interface there. With that interface, I just need to run spearmint-lite.py and it will write the proposed hyperparam choice to a result file. I will then manually run my machine learning application with that proposed hyperparam choice and report back the loss to the result file, and run spearmint-lite.py again to get another proposed hyperparam choice.

If I understand correctly, in this codebase, I can only talk with Spearmint with a callback python script (something like branin.py), and Spearmint will call branin.py to evaluate each hyperparam choice. Since it's very hard for me to launch my machine learning platform with a python script, and I would rather manually run each experiment, I'm wondering is there any way that I can manually run my experiments?

Thank you very much!

After several successful runs, it seems spearmint crashes with the following output:

/Users/name/Desktop/external_libraries/Spearmint/spearmint/tasks/task.py:287: RuntimeWarning: invalid value encountered in subtract
  return y - mean
Fitting GP for main task...
Traceback (most recent call last):
 File "main.py", line 494, in <module>
   main()
 File "main.py", line 286, in main
   suggested_job = get_suggestion(chooser, resource.tasks, db, expt_dir, options, resource_name)
 File "main.py", line 355, in get_suggestion
   hypers = chooser.fit(task_group, hypers, task_options)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/choosers/default_chooser.py", line 309, in fit
   hypers=hypers.get(task_name, None)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/models/gp.py", line 505, in fit
   self._hypers_list = self._collect_samples(self.mcmc_iters)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/models/gp.py", line 378, in _collect_samples
   sampler.sample(self)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/slice_sampler.py", line 262, in sample
   params_array, current_ll = slice_sample(params_array, self.logprob, model, **self.sampler_options)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/mcmc.py", line 354, in slice_sample
   new_x, new_llh = direction_slice(direction, init_x)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/mcmc.py", line 289, in direction_slice
   llh_s = np.log(npr.rand()) + dir_logprob(0.0)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/mcmc.py", line 272, in dir_logprob
   return logprob(direction*z + init_x, *logprob_args)
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/sampling/slice_sampler.py", line 240, in logprob
   lp += model.log_likelihood()
 File "/Users/name/Desktop/external_libraries/Spearmint/spearmint/models/gp.py", line 538, in log_likelihood
   solve = spla.cho_solve((chol, True), self.observed_values - self.mean.value)
 File "/usr/local/lib/python2.7/site-packages/scipy/linalg/decomp_cholesky.py", line 162, in cho_solve
   b1 = asarray_chkfinite(b)
 File "/usr/local/lib/python2.7/site-packages/numpy/lib/function_base.py", line 613, in asarray_chkfinite
   "array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs

My config file is as follows, if helpful:

{
   "language": "PYTHON",
   "main-file": "stub_optimizing.py",
   "experiment-name": "swingy-monkey-optimization",
   "likelihood": "GAUSSIAN",
   "variables": {
       "height_bucket_div": {
           "type": "FLOAT",
           "size": 1,
           "min": 1,
           "max": 15
       },
       "width_bucket_div": {
           "type": "FLOAT",
           "size": 1,
           "min": 1,
           "max": 10
       },
       "vel_bucket_len": {
           "type": "FLOAT",
           "size": 1,
           "min": 1,
           "max": 30
       },
       "discount": {
           "type": "FLOAT",
           "size": 1,
           "min": 0,
           "max": 1
       },
       "greed_exp": {
           "type": "FLOAT",
           "size": 1,
           "min": 1,
           "max": 3
       }
   }
}

Hi, I'm trying to use Spearmint to optimize my hyperparameters for my machine learning task. I have one hyperparameter variable (among many other hyperparameters, say it's X) that stands for the type of my model (SVM, logistic regression, etc.). For example, if the X==0, it's SVM, and if X==1, it's logistic regression, etc.

However, I find if I change the range of X, to make it start from 1, (X==1 means SVM, and X==2 means logistic regression), the behavior of Spearmint becomes quite different (i.e., it proposes quite different hyperparameter choices for me to try).

Intuitively, the actual value range of X shouldn't matter. So I'm wondering is what I see expected, or I did something wrong?

Thank you!

This commit is a direct application of this idea: https://github.com/HIPS/Spearmint/issues/32#issue-95374628

Tested on my own 1D problem and works well.

Do folks either doing dev or just using spearmint at Harvard already have a stable installation set up? If so, can I soft link it or something? Or maybe there's a module built? I haven't found one using module avail, but I suppose I could request one via RC.

Hi I get the following error when running the 'constrained' example using PESC using the latest origin/PESC code. All other examples appear to work fine for me. It appears to do a single fit but then produces errors when trying to identify the next point to search.

I looked into the code and it seems model.options doesn't have a key 'binomial_trials'. Also model._one_minus_epsilon is not defined.

Any help would be appreciated, Thanks.

##@##:~/Spearmint$ python spearmint/main.py examples/constrained/

Getting suggestion for y_at_least_x, y_at_most_10, branin...


Suggestion:     
                NAME          TYPE       VALUE
                ----          ----       -----
                X             float      6.781006    
                Y             float      11.715088   
Submitted job 1 for tasks(s) y_at_least_x, y_at_most_10, branin with local scheduler (process id: 56519).
Current time: 2015-08-02 17:55:27
Status: 1 pending, 0 complete.
ID(s) of pending job(s) for y_at_least_x, y_at_most_10, branin: 1
Waiting for results...

Fitting GP to 0 data for y_at_least_x task...
Fitting GP to 1 data for NaN task...
Fitting GP to 0 data for y_at_most_10 task...
Fitting GP to 0 data for branin task...
Computing current best...

No feasible solution found (yet).

Maximum total probability of satisfying constraints = 0.05659
  Probability of satisfying       y_at_least_x constraint: 0.33181
  Probability of satisfying                NaN constraint: 0.49298
  Probability of satisfying       y_at_most_10 constraint: 0.34597

At location:    
                NAME          TYPE       VALUE
                ----          ----       -----
                X             float      6.781006    
                Y             float      11.715088   
Traceback (most recent call last):
  File "spearmint/main.py", line 514, in <module>
    main()
  File "spearmint/main.py", line 339, in main
    recommendation = chooser.best()
  File "/Users/##/Spearmint/spearmint/choosers/default_chooser.py", line 820, in best
    val_o, loc_o = self.bestObservedConstrained()
  File "/Users/##/Spearmint/spearmint/choosers/default_chooser.py", line 1242, in bestObservedConstrained
    all_constraints_satisfied = np.all([self.constraintSatisfiedAtObservedInputs(c) for c in self.constraints], axis=0)
  File "/Users/##/Spearmint/spearmint/choosers/default_chooser.py", line 1295, in constraintSatisfiedAtObservedInputs
    sat = values/float(model.options['binomial_trials']) >= model._one_minus_epsilon
KeyError: 'binomial_trials'

After seeing Jasper Snoek talk in cambridge I was inspired to try this bayesian optimization package. It seems well written and well commented, but without much documentation. I had a some difficulty trying to understand what all the variables meant and what all the routines were doing. So while I was going through the code in detail I added some documentation for my own benefit. Maybe you guys will appreciate it. But please go over my documentation carefully. The only thing worse then no documentation is incorrect documentation.

I also made some other changes.

1. I did a bit of code cleanup, like removing tabs, removing unused imports, changing 2-space indents to 4-space, shortening long lines, etc.
2. I added a new example for noisy functions. This is simply the branin function with some added noise.
3. I added a file make_plots.py to the simple branin example. Using your README I was able to easily run the example, but I had no idea what to do then. I didn't know how to access the data or make predictions or find the best guess for the minimum, or show uncertainties. make_plots.py is basically my solution to that, though it may be a bit hacky. You may want to update it if there are more elegant ways of doing this. I wrote it in a general way so it can be called from any experiment directory with a 2-dimensional function (e.g. examples/noisy)
4. I restructured main.py, separating out the part that reads in the options. I did this mostly because I wanted to re-use it for make_plots.py
5. I renamed some variables because I found it really hard to know if variable tasks referred to a list of Task objects or a list of task names (strings). This confused me a lot and I only changed a few of the examples.

Finally, thanks for the great code

I want to use spearmint for my research work but I got this error when I tried to imitate the author by running the "python main.py"

Traceback (most recent call last): File "main.py", line 494, in main() File "main.py", line 249, in main options, expt_dir = get_options() File "main.py", line 216, in get_options expt_dir = os.path.realpath(os.path.expanduser(args[0])) IndexError: list index out of range

Again, I also want to Know how I can insert my data into spearmint, I want to optimize hyperparameters for time series regression. Thank you

Hi, I'd like to include spearmint here: https://www.microprediction.com/blog/optimize However can you clarify if people contributing to the open source prediction network can use it? The network is intended for any use at all including commercial use. Peter

Hey, I was wondering in which file should I look for the uncertainty values around the expected value of the objective function (of fitted GP) so that I could print them through iterations? Thanks!

i modified the examples/simple/brainin.py script to test a simple function x^2 + y^2. However, when i run main,py, it seems like it continues to optimize what that function was prior..