The Python ensemble sampling toolkit for affine-invariant MCMC

Dan Foreman-Mackey

Last update: Dec 31, 2022

Related tags

Overview

emcee

The Python ensemble sampling toolkit for affine-invariant MCMC

http://img.shields.io/badge/license-MIT-blue.svg?style=flat

http://img.shields.io/badge/arXiv-1202.3665-orange.svg?style=flat

https://coveralls.io/repos/github/dfm/emcee/badge.svg?branch=master&style=flat&v=2

https://readthedocs.org/projects/emcee/badge/?version=latest

emcee is a stable, well tested Python implementation of the affine-invariant ensemble sampler for Markov chain Monte Carlo (MCMC) proposed by Goodman & Weare (2010). The code is open source and has already been used in several published projects in the Astrophysics literature.

Documentation

Read the docs at emcee.readthedocs.io.

Attribution

Please cite Foreman-Mackey, Hogg, Lang & Goodman (2012) if you find this code useful in your research and add your paper to the testimonials list. The BibTeX entry for the paper is:

@article{emcee,
   author = {{Foreman-Mackey}, D. and {Hogg}, D.~W. and {Lang}, D. and {Goodman}, J.},
    title = {emcee: The MCMC Hammer},
  journal = {PASP},
     year = 2013,
   volume = 125,
    pages = {306-312},
   eprint = {1202.3665},
      doi = {10.1086/670067}
}

License

emcee is free software made available under the MIT License. For details see the LICENSE file.

Comments

Submitting a JOSS paper
I've just pushed a draft of a paper for submission to JOSS in parallel with the release of version 3.0. I'd like to invite members of the community to join as an author on this paper if you're are interested. Please reply to this thread with your full name, affiliation, and (optionally) ORCID in the format needed by JOSS if you would like to join as an author. Pull requests or comments about the manuscript are also welcome! For example, my info is:

name: Daniel Foreman-Mackey orcid: 0000-0003-0872-7098 affiliation: Center for Computational Astrophysics, Flatiron Institute

I'd like to submit this on 28 Oct 2019 so let me know if you're interested in joining, but need more time to decide.

Pinging committers with at least 10 lines (but others are also welcome if you feel that you have contributed!):

@farr

@manodeep

@davidwhogg

@jeremysanders

@eteq

@joezuntz

@pkgw

@andyfaff

@bencebeky

@dstndstn

@drphilmarshall

@willvousden

@terhardt

@Christopher-Bradshaw

@ipashchenko

@OriolAbril

@ericdill

@mindriot101

@nkern

@migueldvb
opened by dfm 34
New example: multimodal.py, to test PTSampler evidence estimation
It's just the example in the docs, plus a little evidence check. Here's the result:

Estimated log evidence = -16.9586413385 +/- 7.29507626509 Analytic log evidence = -6.67931612501

which means that there is at least one bug in this code! But it runs to completion, and the docs are up to date. What more could one want?
opened by drphilmarshall 31
Add seed argument to EnsembleSampler

I saw this come up on the mailing list and had some time for this. I hope it helps.

This PR adds seed as a new argument to EnsembleSampler with default value None. Also, with seed=None the random state is not reseeded but instead it is left as is so that calling np.random.seed() before creating the sampler still works the same way. It also adds a test to check that the seed argument "overwrites" np.random.seed.

opened by OriolAbril 22
Using an ensemble sampler within a Gibbs step
After talking to @davidwhogg about hierarchical modelling and Gibbs sampling at @astrohackweek, I've got some implementation questions. #1. Can ensemble samplers be used in both/all phases of a Gibbs step?

For example, I have ensemble samplers for parameters local to each pixel. Each sampler has N_w walkers. Now how do I pass the last step from the pixel samplers on to the next phase that samples in global parameters? Specifically, what goes in the args to the run_mcmc method? I see two options:

Just take the state from a single walker (the first one? or a random one?) and pass that on to the global sampler.

If the ensemble samplers at the local and global levels both have N_w walkers then I could 'align' the walkers of each. Hence the final state of walker i at the local level would be used by walker i at the global level. I think this is more appealing but is means that I'd need some way of telling the posterior function which walker it belongs to so it can choose the appropriate parameters from the args. @dfm, is there already a way to do this?

Or is it A Bad Idea :tm: to use an ensemble sampler within Gibbs (as I think @dfm alluded in #106, though @davidwhogg seemed optimistic)? #2. How can ensemble samplers be used as a step with a non-ensemble sampler in another step?

If I use an ensemble level for one level of the Gibbs step and a Metropolis-Hastings in another, what parameters do I pass on to MH? Do I always take the parameters at the last step of the i-th walker? Or pick a random walker? (This question mirrors option 1 from above). #3. Is it a special case to have an ensemble sampler step go into a trivial Gaussian Process step?

As @davidwhogg pointed out, in one of my global Gibbs phases I can make a sample from a normal distribution defined by model-observation residuals. In computing these residuals, should I:

marginalize over the models of all walkers to make a single new sample

just pick a single walker to make a single new sample, or

make a new value for each walker (which means that again, the posterior function for the ensemble sampler at the next level would need to know what walker it belongs to).

(This is a question directed mostly to @davidwhogg and @dfm but anyone is welcome to chime in!)
opened by jonathansick 22
ValueError: algorithm must be 'MT19937' when creating sampler
General information:

emcee version: v3.0rc2

platform: Ubuntu 18.04.1 LTS

installation method (pip/conda/source/other?): pip install emcee==3.0rc2

Problem description: Running emcee.EnsembleSampler() throws error.

Expected behavior:

No error, creating the sampler correctly.

Actual behavior:

ValueError: algorithm must be 'MT19937'

What have you tried so far?:

The error is coming from emcee/ensemble.py, line 133. I tried printing state variable, and it seems like the first element of this list, which should be 'MT19937' is in reality b'MT19937'. So it seems like it is a bytes string. As a quick fix I tried setting state[0] = 'MT19937' just before the call to set_state and it seems to work fine, although this solution is obviously no good.

Minimal example:

import emcee # File with posterior class: import posterior as pos posterior = pos.posterior() filename = 'test.chains' name = '000' ndim = 3 nwalkers = 10 max_n = 10000 backend = emcee.backends.HDFBackend(filename, name=name) sampler = emcee.EnsembleSampler(nwalkers, ndim, log_prob_fn=posterior.lnpos, args=[[ns_range[0], ns_range[1]],[]], backend=backend)
Bug
opened by iipr 17
Support for named parameters
This PR closes #385 by giving support for named parameters in the EnsembleSampler.

Specifically this PR:

allows a user to pass in a list of names (types as str but really can be anything) for the parameters to the constructor for EnsembleSampler. Checks are made to ensure that the number of unique names are ndim

In the EnsembleSampler.compute_log_prob method, these names are used to reshape the parameters passed in to the method (having previously been generated by the Move) so that instead of an ndarray of shape (nwalkers, ndim) it becomes a list of length nwalkers of dictionaries of length ndim, with each key-value pair being the names and parameters

A helper function was written to perform the ndarray -> List[Dict[str, np.number]] conversion, so that it could be tested separately from the compute_log_prob method (higher testable surface area)

test of this functionality were placed in the newly created src/emcee/tests/unit/test_ensemble.py file. I tested the helper method, compute_log_prob, and run_mcmc

very small bits of cleanup in ensemble.py to be more PEP8 compliant/more pythonic

As discussed in #385, this PR makes no assumptions about the kinds of likelihood functions or underlying models being utilized. It does not take gradients of the likelihood, nor does it break the API (frontend or backend) in any way. I believe the unit test in in the TestNamedParametes tests make it clear why this is useful. Positional bugs will be a thing of the past!

Support for named parameters could go a bit further. For example:

I didn't allow vectorize=True when naming the parameters. I guess the right thing to allow for would be if vectorize=True then we produce one dictionary where the keys are the parameter names and the values are 1D arrays of each parameter.

I did not implement support for parameter names in the PTSampler. IMO doing this would be "double work" and from a software perspective it would make more sense (IMO) to make an abstract Sampler class that handled the names, and that both the EnsembleSampler and PTSampler would subclass. But, this is obviously a non-trivial refactor (that I would be happy to contribute to).

Thanks @dfm for the discussion on #385. I look forward to your review.
opened by tmcclintock 15
Support longdouble data types

The goal is to have emcee work with coordinates that are numpy long doubles (80-bit precision on most Intel, 128-bit on arm64, 64-bit on MSVC). It's not essential that affine combinations permit long doubles as coefficients; the concern is when the clump of walkers is in a small region around some large-ish long double value. This comes up when fitting times in pulsar data.

opened by aarchiba 15
Add check that `initial_state` allows full exploration of parameter space

When running the ensemble sampler, I occasionally accidentally initialize all walkers to have the same initial position in some of the dimensions. This is entirely my fault! But, it would be easy (I think) to add a check that the initial state allows the sampler to explore all of parameter space, rather than have me find out my mistake after running my chains for a while.

If you think this would be a useful thing to have, I'm happy to make the change and submit a PR, but wanted to check it was something you'd be interested in/I haven't missed some reason why you might not want this.

Thanks!

opened by Christopher-Bradshaw 15

AutocorrError, 'The chain is too short to reliably estimate the autocorrelation time'

I've been using emcee without any problems for a while. Recently I tried running my program in Windows10, but I get an error with the autocorr (that I didn't get before). Using the quickstart.py example I get the same problem but I can't figure out whats wrong whatsoever. I hope you can help me figuring out whats wrong . This is the log I get running quickstart.py :

In [1]: run quickstart.py
Mean acceptance fraction: 0.191032
---------------------------------------------------------------------------
AutocorrError                             Traceback (most recent call last)
C:\Users\anibal\Documents\GitHub\emcee\examples\quickstart.py in <module>()
     58 # Estimate the integrated autocorrelation time for the time series in each
     59 # parameter.
---> 60 print("Autocorrelation time:", sampler.get_autocorr_time())
     61
     62 # Finally, you can plot the projected histograms of the samples using

C:\Users\anibal\Anaconda2\lib\site-packages\emcee-2.2.1-py2.7.egg\emcee\ensemble.pyc in get_autocorr_time(self, low, high, step, c, fast)
    501         return autocorr.integrated_time(np.mean(self.chain, axis=0), axis=0,
    502                                         low=low, high=high, step=step, c=c,
--> 503                                         fast=fast)
    504
    505

C:\Users\anibal\Anaconda2\lib\site-packages\emcee-2.2.1-py2.7.egg\emcee\autocorr.pyc in integrated_time(x, low, high, step, c, full_output, axis, fast)
    116             break
    117
--> 118     raise AutocorrError("The chain is too short to reliably estimate "
    119                         "the autocorrelation time")
    120

AutocorrError: The chain is too short to reliably estimate the autocorrelation time

opened by ReddTea 15

Added support for emcee in arviz

Wanted to let you know we have added support for emcee to the arviz library - see https://github.com/arviz-devs/arviz/pull/302 for a description of the change.

Roughly, the plan is for arviz to replace plotting and diagnostics for pymc3 (so we do not have to rewrite it for pymc4), by converting data to a common data format (netCDF/xarray datasets), and then implementing all the plots and diagnostics on that object. emcee is the third library to have support (pymc3 and pystan), after hearing so many astronomers saying nice things about it at a conference.

In addition to the plots you can see in the pull request above, you can also calculate things like the effective sample size and the gelman rubin statistic. The summary function below shows a number of these.

Some questions I had -

we store sample_stats from pystan and pymc3. Is there a way to access whether a sample was accepted at a per-draw basis in emcee?
Are there particular diagnostics or plots you would like to see?

import arviz as az
from arviz.tests import helpers

sampler = helpers.emcee_linear_model(None, 500, 10)  # this is the sampler from http://dfm.io/emcee/current/user/line/

# This converter turns the data into a netCDF object that can be serialized and shared
data = az.from_emcee(sampler, var_names=['ln(f)', 'b', 'm'])

# all arviz functions should work with this object
az.summary(data).to_dataframe()

	ln(f)	b	m
metric
mean	-0.901523	4.186736	-0.493168
standard deviation	0.083640	0.387793	0.146651
mc error	0.002052	0.008238	0.003727
hpd 3.00%	-1.061138	3.448212	-0.759357
hpd 97.00%	-0.742526	4.923531	-0.201992
effective samples	1429.248983	1408.863122	1287.373414
gelman-rubin statistic	1.030000	1.030000	1.040000

opened by ColCarroll 13

v3.0.0dev0 blobs failing when different shapes
General information:

emcee version: v3.0.0dev0

platform: any

installation method (pip/conda/source/other?): any

Problem description:

Expected behavior:

I expect that returning blobs of various shapes should work.

Actual behavior:

An error is raised: ValueError: setting an array element with a sequence.

What have you tried so far?:

When passing blobs_dtype with shapes for each returned blob, it works fine.

Minimal example:

import emcee import numpy as np def lnl(p): return p[0], 1.0, np.array([1,2,3]) sampler = emcee.EnsembleSampler( nwalkers = 10, ndim = 2, log_prob_fn = lnl ) p0 = np.random.normal(size=(10,2)) for sample in sampler.sample(p0, iterations=10): continue

The offending line is L394 of ensemble.py: dt = np.atleast_1d(blob[0]).dtype. This will work as intended (I think), if it is instead: dt = [(np.atleast_1d(b).dtype, np.atleast_1d(b).shape) for b in blob[0]], though there might need to be a check for 1-dimensional arrays.
Bug
opened by steven-murray 13
My samplers do not converge to maximum likelihood
General information:

emcee version: 3.0.2

platform: Ubuntu Linux

installation method (pip/conda/source/other?): source

Problem description:

Expected behavior: The samplers converge to best-fit parameters that have the maximum likelihood

Actual behavior: corner plot shows that parameters converge to a group of parameters that do not have the maximum likelihood. Besides, best-fit parameters are not covered by [16%,84%] range of samples.

What have you tried so far?: I saved every group of parameters called by "log_likelihood" and the corresponding likelihood value returned by "log_likelihood" into a new file. I run MCMC sampling for 2000 steps with 12 chains and burned 0 steps. I found that "get_chain" returns 24000 samples while my file only saved less than 20000 samples. The samples returned by "get_chain" do not converge while samples saved by my file show that they are converged. When I was checking the likelihood value I found that samples do not converge to the best-fit parameters that gave the maximum likelihood. I guess the reason is that the best-fit parameters are too close to the boundary of parameter space defined in "log_prior". Therefore MCMC converged to a local minimum instead of the global minimum. To enlarge the parameter space would make it unphysical. What should I do?

Minimal example:

import emcee ## I paste my "log_likelihood" here. import numpy as np import os def log_likelihood(par, y ,yerr): p1,p2,p3,p4,p5,p6 = par model = model(p1,p2,p3,p4,p5,p6) likelihood = -0.5*np.sum((y-model)**2/yerr**2) # save parameters called by log_likelihood and likelihood to new files w = [p1,p2,p3,p4,p5,p6,likelihood] pid = os.getpid() f =open(str(pid) + 'csv','a') csv_writer = csv.writer(f,dialect='excel') csv_writer.writerow(w) f.close() return likelihood # sample code goes here...
opened by tga54 0

Parallelization of emcee is working not as fast as expected

General information:

emcee version: 3.1.3
platform: JupyterHub, running Linux
installation method: pip

Problem description:

Expected behavior:

Hi all,

I was following this example for Parallelization using Multiprocessing. From the documentation, the code should run 3.3 times faster in parallel, compared to running in serial, given that we have a 4-core CPU.

Actual behavior:

This didn't happen on my side. When using 8 cores, I only obtained a 1.1x performance boost.

Minimal example:

import emcee
import os
os.environ["OMP_NUM_THREADS"] = "1"
import time
import numpy as np

def log_prob(theta):
    t = time.time() + np.random.uniform(0.005, 0.008)
    while True:
        if time.time() >= t:
            break
    return -0.5 * np.sum(theta**2)

np.random.seed(42)

initial = np.random.randn(32, 5)
nwalkers, ndim = initial.shape
nsteps = 100

sampler = emcee.EnsembleSampler(nwalkers, ndim, log_prob)
start = time.time()
sampler.run_mcmc(initial, nsteps, progress=True)
end = time.time()
serial_time = end - start
print("Serial took {0:.1f} seconds".format(serial_time))

from multiprocessing import Pool

with Pool() as pool:
    sampler = emcee.EnsembleSampler(nwalkers, ndim, log_prob, pool=pool)
    start = time.time()
    sampler.run_mcmc(initial, nsteps, progress=True)
    end = time.time()
    multi_time = end - start
    print("Multiprocessing took {0:.1f} seconds".format(multi_time))
    print("{0:.1f} times faster than serial".format(serial_time / multi_time))

Please see the attachment for the notebook I was using. emceeTest.zip

opened by porpose 4

Issue on class emcee.moves.GaussianMove(cov, mode='vector', factor=None)

For classemcee.moves.GaussianMove(cov, mode='vector', factor=None), the emcee docs (and the comments in the source code) all say the parameter factor (Optional[float]) is invalid for the "vector" mode. However, I do not find factor option is forbidden in actual calculation of proposal samples when mode="vector" in the following code. Also, I don't quite understand mathematically why factor (Optional[float])` is invalid for the "vector" mode.

As far as I understand, factor (Optional[float]) makes the Gaussian with flexible scale (i.e., add a factor to the fixed input scale parameter), which somewhat introduce more randomness to benefit the proposal. Then the proposal distribution turn out to be Here, x is the proposal sample, G represent Gaussian distribution, cov and factor are the input parameters, f is the proposal factor value in actual calculation, which follows LogUniform(-self.log_factor, self.log_factor) in this line.

From my understanding, the proposal distribution keeps a symmetric distribution, which means we do not need to introduce additional factor to account for difference in proposal distribution q(x|x') and q(x'|x) when calculating the acceptance rate in MCMC, i.e., the same as a regular Gaussian proposal. And it's nothing related to mode if I understand correctly.

Also, another question comes to me when thinking of the mode. If one provides the full, square covariance matrix and choose "random", or "sequential" mode, that means, sampler selects one dimension in each iteration, proposal distribution becomes marginalized Gaussian in this situation. Only the variance element will affect the proposal but the covariance (non-diagonal elements) won't be considered. That's probably a loss of information. Maybe "random", or "sequential" mode should be disabled in this case?

I may make some mistakes when thinking of all these. Please correct me if there're some problems. Thanks!

opened by HuiTong5 0
Add `DIMEMove`

This PR adds the DIMEMove as proposed here: https://gregorboehl.com/live/dime_mcmc_boehl.pdf

DIME is an ensemble move that is intended to provide fast burn-in for high-dimensional distributions and to perform well for multimodal distributions. See here for an example: https://github.com/gboehl/emcwrap#tutorial

I am happy to create some documentation, e.g. by adding the above tutorial to the emcee documentation.

I will also post in the mailing list...

Cheers!

opened by gboehl 2

3.1.3: sphinx warnings `reference target not found`

First of all currently it is not possible to use straight sphinx-build command to build documentation out of source tree

+ /usr/bin/sphinx-build -j48 -n -T -b man docs build/sphinx/man
Running Sphinx v5.2.2
/usr/lib/python3.8/site-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 1.17.1-unknown is an invalid version and will not be supported in a future release
  warnings.warn(
making output directory... done
myst v0.18.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=['dollarmath', 'colon_fence'], disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, footnote_transition=True, words_per_minute=200, sub_delimiters=('{', '}'), linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area')
myst-nb v0.16.0: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, execution_mode='off', execution_cache_path='', execution_excludepatterns=(), execution_timeout=-1, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/build/sphinx/.jupyter_cache
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 15 added, 0 changed, 0 removed
reading sources... [100%] user/upgrade
WARNING: autodoc: failed to import function 'autocorr.integrated_time' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import function 'autocorr.function_1d' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'backends.Backend' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'backends.HDFBackend' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.RedBlueMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.StretchMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.WalkMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.KDEMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.DEMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.DESnookerMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.MHMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'moves.GaussianMove' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'EnsembleSampler' from module 'emcee'; the following exception was raised:
No module named 'emcee'
WARNING: autodoc: failed to import class 'State' from module 'emcee'; the following exception was raised:
No module named 'emcee'
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... python-emcee.3 { user/install user/sampler user/moves user/blobs user/backends user/autocorr user/upgrade user/faq tutorials/quickstart tutorials/line tutorials/parallel tutorials/autocorr tutorials/monitor tutorials/moves } /home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/sampler.rst:6: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/sampler.rst:12: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/sampler.rst:12: WARNING: py:class reference target not found: State
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:27: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.StretchMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.RedBlueMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.MHMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.GaussianMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/blobs.rst:59: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/backends.rst:12: WARNING: py:class reference target not found: emcee.backends.HDFBackend
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/backends.rst:12: WARNING: py:class reference target not found: backends.HDFBackend
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/upgrade.rst:19: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:70002: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:120002: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:180002: WARNING: py:func reference target not found: EnsembleSampler.run_mcmc
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:180002: WARNING: py:func reference target not found: EnsembleSampler.reset
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:200002: WARNING: py:func reference target not found: EnsembleSampler.get_chain
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:220002: WARNING: py:func reference target not found: EnsembleSampler.acceptance_fraction
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/line.ipynb:190002: WARNING: py:func reference target not found: EnsembleSampler.get_chain
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/monitor.ipynb:10009: WARNING: py:class reference target not found: backends.HDFBackend
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/monitor.ipynb:110002: WARNING: py:class reference target not found: backends.HDFBackend
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/monitor.ipynb:130004: WARNING: py:func reference target not found: backends.HDFBackend.reset
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:40002: WARNING: py:class reference target not found: moves.StretchMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:80002: WARNING: py:class reference target not found: moves.DEMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:80002: WARNING: py:class reference target not found: moves.DESnookerMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:80002: WARNING: py:class reference target not found: moves.DESnookerMove
done
build succeeded, 40 warnings.

This can be fixed by patch like below:

--- a/docs/conf.py
+++ b/docs/conf.py
@@ -1,5 +1,9 @@
 # -*- coding: utf-8 -*-

+import sys
+import os
+sys.path.insert(0, os.path.abspath("../src"))
+
 from pkg_resources import DistributionNotFound, get_distribution

 try:

This patch fixes what is in the comment and that can of fix is suggested in sphinx example copy.py https://www.sphinx-doc.org/en/master/usage/configuration.html#example-of-configuration-file

Than .. on building my packages I'm using sphinx-build command with -n switch which shows warmings about missing references. These are not critical issues.

+ /usr/bin/sphinx-build -j48 -n -T -b man docs build/sphinx/man
Running Sphinx v5.2.2
making output directory... done
myst v0.18.0: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=['dollarmath', 'colon_fence'], disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, footnote_transition=True, words_per_minute=200, sub_delimiters=('{', '}'), linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area')
myst-nb v0.16.0: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, execution_mode='off', execution_cache_path='', execution_excludepatterns=(), execution_timeout=-1, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/build/sphinx/.jupyter_cache
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 15 added, 0 changed, 0 removed
reading sources... [100%] user/upgrade
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... python-emcee.3 { user/install user/sampler user/moves user/blobs user/backends user/autocorr user/upgrade user/faq tutorials/quickstart tutorials/line tutorials/parallel tutorials/autocorr tutorials/monitor tutorials/moves } /home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/sampler.rst:6: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler:1: WARNING: py:class reference target not found: callable
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler:18: WARNING: py:class reference target not found: StretchMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_autocorr_time:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_autocorr_time:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_blobs:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_blobs:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_chain:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_chain:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_chain:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_log_prob:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.get_log_prob:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.sample:: WARNING: py:class reference target not found: ndarray
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.sample:: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.sample:: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/ensemble.py:docstring of emcee.ensemble.EnsembleSampler.sample:: WARNING: py:class reference target not found: NoneType
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/sampler.rst:12: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/sampler.rst:12: WARNING: py:class reference target not found: State
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/state.py:docstring of emcee.state.State:1: WARNING: py:class reference target not found: ndarray
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/state.py:docstring of emcee.state.State:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/state.py:docstring of emcee.state.State:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/state.py:docstring of emcee.state.State:1: WARNING: py:class reference target not found: ndarray
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/state.py:docstring of emcee.state.State:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/state.py:docstring of emcee.state.State:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:27: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.StretchMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.RedBlueMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.MHMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/moves.rst:33: WARNING: py:class reference target not found: moves.GaussianMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/moves/mh.py:docstring of emcee.moves.mh.MHMove:3: WARNING: py:class reference target not found: moves.GaussianMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/blobs.rst:59: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/backends.rst:12: WARNING: py:class reference target not found: backends.HDFBackend
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_autocorr_time:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_autocorr_time:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_blobs:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_blobs:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_chain:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_chain:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_chain:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_log_prob:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.get_log_prob:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.save_step:3: WARNING: py:class reference target not found: State
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/backend.py:docstring of emcee.backends.backend.Backend.save_step:1: WARNING: py:class reference target not found: ndarray
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.hdf.HDFBackend:1: WARNING: py:class reference target not found: str; optional
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.hdf.HDFBackend:1: WARNING: py:class reference target not found: bool; optional
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_autocorr_time:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_autocorr_time:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_blobs:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_blobs:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_chain:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_chain:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_chain:1: WARNING: py:class reference target not found: ndim
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_log_prob:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.backend.Backend.get_log_prob:1: WARNING: py:class reference target not found: nwalkers
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.hdf.HDFBackend.save_step:3: WARNING: py:class reference target not found: State
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/backends/hdf.py:docstring of emcee.backends.hdf.HDFBackend.save_step:1: WARNING: py:class reference target not found: ndarray
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/autocorr.py:docstring of emcee.autocorr.integrated_time:16: WARNING: py:class reference target not found: AutocorrError
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/autocorr.py:docstring of emcee.autocorr.integrated_time:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/src/emcee/autocorr.py:docstring of emcee.autocorr.function_1d:1: WARNING: py:class reference target not found: array
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/user/upgrade.rst:19: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:70002: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:120002: WARNING: py:class reference target not found: EnsembleSampler
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:180002: WARNING: py:func reference target not found: EnsembleSampler.run_mcmc
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:180002: WARNING: py:func reference target not found: EnsembleSampler.reset
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:200002: WARNING: py:func reference target not found: EnsembleSampler.get_chain
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/quickstart.ipynb:220002: WARNING: py:func reference target not found: EnsembleSampler.acceptance_fraction
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/line.ipynb:190002: WARNING: py:func reference target not found: EnsembleSampler.get_chain
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/monitor.ipynb:10009: WARNING: py:class reference target not found: backends.HDFBackend
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/monitor.ipynb:110002: WARNING: py:class reference target not found: backends.HDFBackend
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/monitor.ipynb:130004: WARNING: py:func reference target not found: backends.HDFBackend.reset
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:40002: WARNING: py:class reference target not found: moves.StretchMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:80002: WARNING: py:class reference target not found: moves.DEMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:80002: WARNING: py:class reference target not found: moves.DESnookerMove
/home/tkloczko/rpmbuild/BUILD/emcee-3.1.3/docs/tutorials/moves.ipynb:80002: WARNING: py:class reference target not found: moves.DESnookerMove
done
build succeeded, 74 warnings.

You can peak on fixes that kind of issues in other projects https://github.com/latchset/jwcrypto/pull/289 https://github.com/click-contrib/sphinx-click/commit/abc31069 https://github.com/latchset/jwcrypto/pull/289 https://github.com/RDFLib/rdflib-sqlalchemy/issues/95 https://github.com/sissaschool/elementpath/commit/bf869d9e https://github.com/jaraco/cssutils/issues/21 https://github.com/pywbem/pywbem/pull/2895 https://github.com/sissaschool/xmlschema/commit/42ea98f2 https://github.com/RDFLib/rdflib/pull/2036 https://github.com/frostming/unearth/issues/14 https://github.com/pypa/distlib/commit/98b9b89f

opened by kloczek 12

Releases(v3.1.3)

v3.1.3(Sep 27, 2022)
What's Changed

Update link to Goodman & Weare 2010 by @covertg in https://github.com/dfm/emcee/pull/432

Pass on random generator to KDE resample method by @hendriko373 in https://github.com/dfm/emcee/pull/436

Fix longdouble tests by seeding np.random by @dfm in https://github.com/dfm/emcee/pull/439

New Contributors

@covertg made their first contribution in https://github.com/dfm/emcee/pull/432

@hendriko373 made their first contribution in https://github.com/dfm/emcee/pull/436

Full Changelog: https://github.com/dfm/emcee/compare/v3.1.2...v3.1.3
Source code(tar.gz)
Source code(zip)
v3.1.3rc1(Sep 27, 2022)

Source code(tar.gz)
Source code(zip)
v3.1.2(May 10, 2022)
What's Changed

add GitHub URL for PyPi by @andriyor in https://github.com/dfm/emcee/pull/419

Make the sample state indexable by @dfm in https://github.com/dfm/emcee/pull/425

Remove NumPy from SETUP_REQUIRES by @BastianZim in https://github.com/dfm/emcee/pull/427

New Contributors

@andriyor made their first contribution in https://github.com/dfm/emcee/pull/419

@BastianZim made their first contribution in https://github.com/dfm/emcee/pull/427

Full Changelog: https://github.com/dfm/emcee/compare/v3.1.1...v3.1.2
Source code(tar.gz)
Source code(zip)
v3.1.1(Aug 23, 2021)
3.1.1 (2021-08-23)

Added support for a progress bar description #401

Source code(tar.gz)
Source code(zip)
v3.1.1rc1(Aug 23, 2021)

Source code(tar.gz)
Source code(zip)
v3.1.0(Jun 25, 2021)
Added preliminary support for named parameters #386

Improved handling of blob dtypes #363

Fixed various small bugs and documentation issues

Source code(tar.gz)
Source code(zip)
v3.1.0rc1(Jun 25, 2021)

A release candidate for v3.1.0
Source code(tar.gz)
Source code(zip)
v3.0.2(Nov 15, 2019)
Added tutorial for moves interface

Added information about contributions to documentation

Improved documentation for installation and testing

Fixed dtype issues and instability in linear dependence test

Final release for JOSS submission

Source code(tar.gz)
Source code(zip)
v3.0.1(Oct 28, 2019)
Added support for long double dtypes

Prepared manuscript to submit to JOSS

Improved packaging and release infrastructure

Fixed bug in initial linear dependence test

Source code(tar.gz)
Source code(zip)
v3.0.0(Sep 30, 2019)
Added progress bars using tqdm.

Added HDF5 backend using h5py.

Added new Move interface for more flexible specification of proposals.

Improved autocorrelation time estimation algorithm.

Switched documentation to using Jupyter notebooks for tutorials.

More details can be found on the docs: https://emcee.readthedocs.io/en/v3.0.0/user/upgrade.

Source code(tar.gz)
Source code(zip)
v3.0rc2(Sep 27, 2018)

Source code(tar.gz)
Source code(zip)
v3.0rc1(Jun 25, 2018)

A release candidate for the refactored emcee version 3.0.
Source code(tar.gz)
Source code(zip)
v2.2.1(Jul 15, 2016)

Source code(tar.gz)
Source code(zip)