zeus is a Python implementation of the Ensemble Slice Sampling method.

Hi, I'm trying to fit a model using Zeus, however I don't have an analytic model, which forces me to interpolate my model data. I run into issues with the sampler when the walkers explore the parameter space too far from where I expect the MAP to be. For instance, I define one parameter to have a uniform prior in (0.8, 1.2) but when the walkers explore ~1.5 my model interpolation breaks (above interpolation bounds). I have tried filling the absent values (outside interpolation range) with nans or infs but those seem to be problematic. Extrapolation also seems to be problematic too since I get

RuntimeError: Number of expansions exceeded maximum limit! 
Make sure that the pdf is well-defined. 
Otherwise increase the maximum limit (maxiter=10^4 by default).

But this could be other bug and still I doubt extrapolating is a good approach. In short, do you have some way to constrain walkers to a certain domain in the parameter space?

Thanks in advance,

I am trying to reproduce manual working example. But it gives me the following error: AttributeError: module 'zeus' has no attribute 'EnsembleSampler'

But it works with emcee though!

Is there a particular reason why the stepping out is done in linear steps rather than doubling the step width (as in other implementations)? I am running into the limit (1e4) when testing on a 100d gaussian with std ranging from 1e-1 ... 1e-9, and increasing the limit did not solve this.

I am working with a problem where some points in parameter space give invalid inputs for which my likelihood fails. When this happens, I would like zeus to print the parameter values at the point that caused the failure (as is done by emcee) to make it easier to debug/understand the physical reason for the likelihood failure.

Can you implement this please?

Currently I think that if you resume a chain (by passing the result of get_last_sample to a new call to run_mcmc) then time is wasted recomputing the posteriors of the starting points, which were already computed last time.

Emcee lets you optionally pass in the posteriors of the starting point, if they are known. Would this be possible for zeus?

This is a minor issue, as I can work around the difference using get_chain instead.

In emcee, get_last_sample is a function which returns the last sample. In zeus this is a property. For compatibility, it would be useful to match the emcee API. It also doesn't make sense to me to use a property called get_XXX.

PS Thanks for working on this software and algorithm - initial tests are looking good

I used a pip install. Then tried to follow the example here: https://zeus-mcmc.readthedocs.io/en/latest/

It didn't work. But changing EnsembleSampler to sampler worked.

I am looking to make it so that I can cancel a Python script mid-run and then resume the job. With Emcee, I do this via the backend feature.

I want to have access to all the previous chains and samples so that I can use previously computed autocorrelation times, and whatnot. So simply passing the previous chains to a new instance of the sampler doesn't quite cut it.

Does Zeus have a specific feature that support resuming jobs? If not, no problem, I can achieve the desired effect via pickling the EnsembleSampler (I hope!). couldn't find anything in the docs, but figured I'd ask before writing code to do this.

Hi @minaskar,

Great work here! Thanks for creating zeus.

I am attaching a pull request that gives some emcee compatiblity, making it easier to use the samplers interchangably.

Some changes:

• store log probability in samples, so they can be retrieved later
• add some functions supported by emcee to allow interchangable runs
• make samples axis order consistent with emcee

I just saw that this also addresses issue #1.

I hope these changes are of interest to you.

Cheers, Johannes

Github recently released a new feature where repository owners can add a CITATION.cff file making it easy for others to cite the repository. Adding a CITATION.cff would make the attribution process very easy for others (myself included 😅 ) who want to cite this work.

Hi, thanks for this great sampler! I was having trouble getting my sampling to work with MPI, everything seemed to run fine, but I got no output. The reason was that my likelihood function was unpickleable, but that was a bit hard to track down, because the pickling errors were suppressed by the ChainManager. So here is a PR to no longer suppress these exceptions. Or if there is a good reason for it, or a better solution, then feel free to close this.

The requirement sklearn is depreciated, and causes issue when installing zeus:

  DEPRECATION: sklearn is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559
  Running setup.py install for sklearn: started
  Running setup.py install for sklearn: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Running setup.py install for sklearn did not run successfully.
  │ exit code: 1
  ╰─> [18 lines of output]
      The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
      rather than 'sklearn' for pip commands.
      
      Here is how to fix this error in the main use cases:
      - use 'pip install scikit-learn' rather than 'pip install sklearn'
      - replace 'sklearn' by 'scikit-learn' in your pip requirements files
        (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
      - if the 'sklearn' package is used by one of your dependencies,
        it would be great if you take some time to track which package uses
        'sklearn' instead of 'scikit-learn' and report it to their issue tracker
      - as a last resort, set the environment variable
        SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error
      
      More information is available at
      https://github.com/scikit-learn/sklearn-pypi-package
      
      If the previous advice does not cover your use case, feel free to report it at
      https://github.com/scikit-learn/sklearn-pypi-package/issues/new
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> sklearn

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.
Error: Process completed with exit code 1.

I can fix this myself for what I'm doing, but advise you change the requirement to scitkit-learn so other users dont have issuess.

I found that running zeus with n_MPI = n_walker/2 gives poor efficiency, with half of the CPU time being spend idling.

If I understand the code in EnsembleSampler.sample correctly, the stepping-out procedure it repeated until all walkers in the ensemble have reached their step-out position, and only then the shrinking procedure begins. This means that the ensemble has to wait until the last walker reaches the step-out position, during which all other walkers are idling. Please correct me if I misunderstood the implementation.

Since the the stepping-out and shrinking procedures are independent for each walker once the directions are set, it should be possible to restructure the loops such that walkers can start shrinking as soon as they finished stepping out, rather than having to wait for the last walker.

On a somewhat unrelated note, is there a reason the maxsteps are distributed randomly between left and right here: https://github.com/minaskar/zeus/blob/master/zeus/ensemble.py#L566 ?

When calling zeus from a script/package that does its own logging, the fact that zeus relies on the root logger is somewhat vexing as it may result in duplicate logging. This commit exposes the option to pass an existing logger to the sampler so that the user can modify logging behaviour if they so desire, or capture zeus' log output in the same streams as other logs. This also enables the user to prevent zeus from clobbering handlers that have already been defined for the root logger.

The changes do not alter the behaviour of existing code - the default of None ensures that any existing code will follow the current logic. While there are other ways of avoiding duplicate logs, this does provide more customisation options to users at rather low cost to maintain.

In autocorr.py, scipy is imported as: 'import scipy as sp' The function _autocorr_func_1d uses scipy for the fft function. However, in newer versions of scipy, you must call the fft function in the following way or else an AttributeError will occur: from scipy import fft fft.fft() # how to use the fft function

Please change this soon. Thank you.

First of all, thanks for the package and all your hard work!

I think I've encountered a couple of issues/bugs in the computation of the R-hat statistic.

1. First is just a typo I think. In lines 139-140 the chain means and variances are flattened by the list comprehension, where I think something like:

_means = np.vstack(means)
_vars = np.vstack(vars)

will keep the structure of the chains, so that np.var(means, ddof=1, axis=0) and np.mean(_vars, axis=0) will give the between-chain and within-chain variance, respectively, across all parameters (right now they end up as scalars, since the _means and _vars are flat).

2. This is similar in spirit to Issue #22 , but with a more significant effect here: on lines 120-121 where each split is reshaped to (-1, ndim), samples from all walkers are collapsed into each split, homogenizing them and leading to unrealistically low R-hat values. In my case I had ~28 walkers, many of which were stuck in well-separated modes and barely mixing at all, but nonetheless had a quite low split R-hat as recorded by the callback because each of the two splits had samples from all 28 walkers, making them statistically similar.

   Since this is an ensemble method I had to spend some time convincing myself, but I really think R-hat should be computed across all (possibly split) walkers, rather than by grouping them together. I can share my trace plots and make a case for this in more detail if it's helpful. With change (1) above fixing this would just be a matter of removing the reshape operation. Then nsplits would determine how many splits are made within each walker.

Thanks again, and let me know what your thoughts are. I'm happy to help implement these changes, too.

...\anaconda3\envs\zeus\Scripts\zeus.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working from collections import Iterable as IterableType

It seems this should become "from collections.abc import Iterable as IterableType"

However, I could not find zeus.py in the repository, so I could not make a pull request to fix this.

I imagine this is likely to become in an error in the next year or so, if not fixed.