pyhsmm MITpyhsmm - Bayesian inference in HSMMs and HMMs. MIT

Related tags

Data Analysis pyhsmm
Overview

Build Status

Bayesian inference in HSMMs and HMMs

This is a Python library for approximate unsupervised inference in Bayesian Hidden Markov Models (HMMs) and explicit-duration Hidden semi-Markov Models (HSMMs), focusing on the Bayesian Nonparametric extensions, the HDP-HMM and HDP-HSMM, mostly with weak-limit approximations.

There are also some extensions:

Installing from PyPI

Give this a shot:

pip install pyhsmm

You may need to install a compiler with -std=c++11 support, like gcc-4.7 or higher.

To install manually from the git repo, you'll need cython. Then try this:

python setup.py install

It might also help to look at the travis file to see how to set up a working install from scratch.

Running

See the examples directory.

For the Python interpreter to be able to import pyhsmm, you'll need it on your Python path. Since the current working directory is usually included in the Python path, you can probably run the examples from the same directory in which you run the git clone with commands like python pyhsmm/examples/hsmm.py. You might also want to add pyhsmm to your global Python path (e.g. by copying it to your site-packages directory).

A Simple Demonstration

Here's how to draw from the HDP-HSMM posterior over HSMMs given a sequence of observations. (The same example, along with the code to generate the synthetic data loaded in this example, can be found in examples/basic.py.)

Let's say we have some 2D data in a data.txt file:

$ head -5 data.txt
-3.711962552600095444e-02 1.456401745267922598e-01
7.553818775915704942e-02 2.457422192223903679e-01
-2.465977987699214502e+00 5.537627981813508793e-01
-7.031638516485749779e-01 1.536468304146855757e-01
-9.224669847039665971e-01 3.680035337673161489e-01

In Python, we can plot the data in a 2D plot, collapsing out the time dimension:

import numpy as np
from matplotlib import pyplot as plt

data = np.loadtxt('data.txt')
plt.plot(data[:,0],data[:,1],'kx')

2D data

We can also make a plot of time versus the first principal component:

from pyhsmm.util.plot import pca_project_data
plt.plot(pca_project_data(data,1))

Data first principal component vs time

To learn an HSMM, we'll use pyhsmm to create a WeakLimitHDPHSMM instance using some reasonable hyperparameters. We'll ask this model to infer the number of states as well, so we'll give it an Nmax parameter:

import pyhsmm
import pyhsmm.basic.distributions as distributions

obs_dim = 2
Nmax = 25

obs_hypparams = {'mu_0':np.zeros(obs_dim),
                'sigma_0':np.eye(obs_dim),
                'kappa_0':0.3,
                'nu_0':obs_dim+5}
dur_hypparams = {'alpha_0':2*30,
                 'beta_0':2}

obs_distns = [distributions.Gaussian(**obs_hypparams) for state in range(Nmax)]
dur_distns = [distributions.PoissonDuration(**dur_hypparams) for state in range(Nmax)]

posteriormodel = pyhsmm.models.WeakLimitHDPHSMM(
        alpha=6.,gamma=6., # better to sample over these; see concentration-resampling.py
        init_state_concentration=6., # pretty inconsequential
        obs_distns=obs_distns,
        dur_distns=dur_distns)

(The first two arguments set the "new-table" proportionality constant for the meta-Chinese Restaurant Process and the other CRPs, respectively, in the HDP prior on transition matrices. For this example, they really don't matter at all, but on real data it's much better to infer these parameters, as in examples/concentration_resampling.py.)

Then, we add the data we want to condition on:

posteriormodel.add_data(data,trunc=60)

The trunc parameter is an optional argument that can speed up inference: it sets a truncation limit on the maximum duration for any state. If you don't pass in the trunc argument, no truncation is used and all possible state duration lengths are considered. (pyhsmm has fancier ways to speed up message passing over durations, but they aren't documented.)

If we had multiple observation sequences to learn from, we could add them to the model just by calling add_data() for each observation sequence.

Now we run a resampling loop. For each iteration of the loop, all the latent variables of the model will be resampled by Gibbs sampling steps, including the transition matrix, the observation means and covariances, the duration parameters, and the hidden state sequence. We'll also copy some samples so that we can plot them.

models = []
for idx in progprint_xrange(150):
    posteriormodel.resample_model()
    if (idx+1) % 10 == 0:
        models.append(copy.deepcopy(posteriormodel))

Now we can plot our saved samples:

fig = plt.figure()
for idx, model in enumerate(models):
    plt.clf()
    model.plot()
    plt.gcf().suptitle('HDP-HSMM sampled after %d iterations' % (10*(idx+1)))
    plt.savefig('iter_%.3d.png' % (10*(idx+1)))

Sampled models

I generated these data from an HSMM that looked like this:

Randomly-generated model and data

So the posterior samples look pretty good!

A convenient shortcut to build a list of sampled models is to write

model_samples = [model.resample_and_copy() for itr in progprint_xrange(150)]

That will build a list of model objects (each of which can be inspected, plotted, pickled, etc, independently) in a way that won't duplicate data that isn't changed (like the observations or hyperparameter arrays) so that memory usage is minimized. It also minimizes file size if you save samples like

import cPickle
with open('sampled_models.pickle','w') as outfile:
    cPickle.dump(model_samples,outfile,protocol=-1)

Extending the Code

To add your own observation or duration distributions, implement the interfaces defined in basic/abstractions.py. To get a flavor of the style, see pybasicbayes.

References

@article{johnson2013hdphsmm,
    title={Bayesian Nonparametric Hidden Semi-Markov Models},
    author={Johnson, Matthew J. and Willsky, Alan S.},
    journal={Journal of Machine Learning Research},
    pages={673--701},
    volume={14},
    month={February},
    year={2013},
}

Authors

Matt Johnson, Alex Wiltschko, Yarden Katz, Chia-ying (Jackie) Lee, Scott Linderman, Kevin Squire, Nick Foti.

Comments
  • pyhsmm on windows

    pyhsmm on windows

    Hi, After a few days of various combinations of python environments and difficulties with the pyhsmm install, I thought I would check in to see if I there was something I could do that would be more successful. Here's the steps I've taken:

    1. Install Anaconda-2.0.1-Windows-x86_64
    2. Download pyhsmm
    3. Install pyhsmm: python setup.py build_ext --inplace
    4. Errors during compile of first cpp file: hmm_messages_interface.cpp: "::hypot not defined"
    5. From http://bugs.python.org/issue11566: add #include cmath earlier than python.h, i.e., at the top of hmm_messages_interface.cpp
    6. "::hypot" error no longer appears, but the next one remains: "error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive]"

    I don't see a quick fix to that one in the searches that I've done.

    Below is the full install console output. You will notice that I am using the gcc that comes with Anaconda (4.7.0). You do suggest using 4.8 and a quick check of using that instead gave the same errors.

    Any thoughts would be most appreciated, Phil

    running build_ext building 'internals.hmm_messages_interface' extension D:\Anaconda\Scripts\gcc.bat -DMS_WIN64 -mdll -O -Wall -Ideps/Eigen3/ -ID:\Anaconda\lib\site-packages\numpy\core\include -ID:\Anaconda\include -ID:\Anaconda\PC -c internals\hmm_messages_interface.cpp -o build\temp.win-amd64-2.7\Release\internals\hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP -DTEMPS_ON_STACK In file included from D:\Anaconda\lib\site-packages\numpy\core\include/numpy/ndarraytypes.h:1761:0, from D:\Anaconda\lib\site-packages\numpy\core\include/numpy/ndarrayobject.h:17, from D:\Anaconda\lib\site-packages\numpy\core\include/numpy/arrayobject.h:4, from internals\hmm_messages_interface.cpp:351: D:\Anaconda\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h:13:79: note: #pragma message: D:\Anaconda\lib\site-packages\numpy\core\include/numpy/npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION In file included from d:\anaconda\mingw\bin../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/include/c++/4.7.0/complex:46:0, from deps/Eigen3/Eigen/Core:56, from internals\hmm_messages.h:4, from internals\hmm_messages_interface.cpp:359: d:\anaconda\mingw\bin../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../../x86_64-w64-mingw32/include/c++/4.7.0/cmath:1096:11: error: '::hypot' has not been declared internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_68sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:10987:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_70sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:11265:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_72sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:11543:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_74sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:11821:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_76sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12099:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_78sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12377:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_80sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12655:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_82sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:12933:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_84sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:13211:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_86sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:13489:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_88sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:13767:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_90sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14045:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_92sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14323:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_94sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14601:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_96sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:14879:865: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_98sample_forwards_log(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:15157:874: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:267:17: error: initializing argument 7 of 'static void hmmc<FloatType, IntType>::sample_forwards_log(int, int, FloatType, FloatType, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_266sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:44761:645: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_268sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:45004:650: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_270sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:45247:645: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_272sample_backwards_normalized(PyObject, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:45490:650: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:277:17: error: initializing argument 5 of 'static void hmmc<FloatType, IntType>::sample_backwards_normalized(int, int, FloatType, FloatType, IntType, FloatType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_286viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:50903:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_288viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51099:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_290viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51295:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_292viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51491:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_294viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51687:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_296viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:51883:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_298viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:52079:602: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = float; IntType = int]' [-fpermissive] internals\hmm_messages_interface.cpp: In function 'PyObject* _pyx_pf_6pyhsmm_9internals_22hmm_messages_interface_300viterbi(PyObject, _Pyx_memviewslice, Pyx_memviewslice, Pyx_memviewslice, PyArrayObject)': internals\hmm_messages_interface.cpp:52275:608: error: invalid conversion from 'pyx_t_5numpy_int32_t* {aka long int}' to 'int' [-fpermissive] In file included from internals\hmm_messages_interface.cpp:359:0: internals\hmm_messages.h:282:17: error: initializing argument 6 of 'static void hmmc<FloatType, IntType>::viterbi(int, int, FloatType, FloatType, FloatType, IntType) [with FloatType = double; IntType = int]' [-fpermissive] error: command 'D:\Anaconda\Scripts\gcc.bat' failed with exit status 1

    opened by pawarrick 28
  • Question on parallelism

    Question on parallelism

    Hi, A quick question on the parallelism implemented for you code. i am not seen much speed up on some test data so I am wondering if I am understanding how to exploit properly your parallel implementation.

    i am starting the ipcluster with $ ipcluster start --n=8 2013-04-02 14:04:54,751.751 [IPClusterStart] Using existing profile dir: u'/home/inti/.config/ipython/profile_default' 2013-04-02 14:04:54.754 [IPClusterStart] Starting ipcluster with [daemon=False] 2013-04-02 14:04:54.754 [IPClusterStart] Creating pid file: /home/inti/.config/ipython/profile_default/pid/ipcluster.pid 2013-04-02 14:04:54.754 [IPClusterStart] Starting Controller with LocalControllerLauncher 2013-04-02 14:04:55.754 [IPClusterStart] Starting 8 Engines with LocalEngineSetLauncher 2013-04-02 14:05:26.478 [IPClusterStart] Engines appear to have started successfully

    Is the parallelism being applied to each data loaded to the model? i.e., each data loaded with model.add_data_parallel() is being run on a different ipython engine.

    Or is the actual state sampling calculation being spread across the engines?

    thanks in advance,

    BW, Inti

    opened by inti 25
  • import error for pyhsmm examples

    import error for pyhsmm examples

    Hey Matt, thanks for your work here.

    I'm trying to run some of the examples and I'm getting ImportError: No module named hmm_messages_interface.

    Full trace is here:


    python hmm.py
    /Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/transitions.py:16: UserWarning: using slow transition counting
      warn('using slow transition counting')
    
    This demo shows how HDP-HMMs can fail when the underlying data has state
    persistence without some kind of temporal regularization (in the form of a
    sticky bias or duration modeling): without setting the number of states to be
    the correct number a priori, lots of extra states can be intsantiated.
    
    BUT the effect is much more relevant on real data (when the data doesn't exactly
    fit the model). Maybe this demo should use multinomial emissions...
    
    Traceback (most recent call last):
      File "hmm.py", line 64, in <module>
        posteriormodel.resample_model()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/models.py", line 431, in resample_model
        self.resample_states(num_procs=num_procs)
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/models.py", line 456, in resample_states
        s.resample()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 356, in resample
        return self.resample_normalized()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 352, in resample_normalized
        alphan = self.messages_forwards_normalized()
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 342, in messages_forwards_normalized
        self._messages_forwards_normalized(self.trans_matrix,self.pi_0,self.aBl)
      File "/Users/daway/Documents/Princeton/Thesis/seldon/lib/python2.7/site-packages/pyhsmm-0.1.5-py2.7-macosx-10.10-intel.egg/pyhsmm/internals/hmm_states.py", line 605, in _messages_forwards_normalized
        from hmm_messages_interface import messages_forwards_normalized
    ImportError: No module named hmm_messages_interface
    

    I think this must be related to an issue I had running python setup.py install. While I am able to import pyhsmm, the install command had an error in it:

    building 'pyhsmm.internals.hmm_messages_interface' extension
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -Ideps/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/usr/local/lib/python2.7/site-packages/numpy/core/include -c pyhsmm/internals/hmm_messages_interface.cpp -o build/temp.macosx-10.10-intel-2.7/pyhsmm/internals/hmm_messages_interface.o -std=c++11 -O3 -w -DNDEBUG -DHMM_TEMPS_ON_HEAP
    pyhsmm/internals/hmm_messages_interface.cpp:5484:672: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_20, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:273:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:5875:680: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_20, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:273:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:9632:452: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_17, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:283:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:9988:456: error: cannot initialize a
          parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_17, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:283:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:10818:442: error: cannot initialize
          a parameter of type 'int *' with an rvalue of type '__pyx_t_5numpy_int32_t
          *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_6, __pyx_pybuffernd_stateseq.diminfo[0].strides))));
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:288:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq)
                         ^
    pyhsmm/internals/hmm_messages_interface.cpp:11014:448: error: cannot initialize
          a parameter of type 'int *' with an rvalue of type '__pyx_t_5numpy_int32_t
          *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_6, __pyx_pybuffernd_stateseq.diminfo[0].strides))));
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    pyhsmm/internals/hmm_messages.h:288:22: note: passing argument to parameter
          'stateseq' here
                IntType *stateseq)
                         ^
    6 errors generated.
    setup.py:35: UserWarning: Failed to build extension modules
      warn('Failed to build extension modules')
    

    For gcc I have

    gcc -v
    Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
    Apple LLVM version 7.0.0 (clang-700.0.72)
    Target: x86_64-apple-darwin14.5.0
    Thread model: posix
    

    Is there something else I need to do with python setup.py install? Thanks!

    opened by dchouren 20
  • problem compiling pyhsmm on Mac OS X 10.9.4

    problem compiling pyhsmm on Mac OS X 10.9.4

    Hi,

    I'm having trouble compiling pyhsmm on Mac OS X 10.9.4, using the default Apple clang compiler (which if I understood correctly the README says it is supported, although g++ is favored).

    I recursively cloned pyhsmm and then installed it in my local virtualenv, using:

    $ python setup.py build_ext --inplace
    

    which results in:

    $ python setup.py build_ext --inplace
    running build_ext
    building 'internals.hmm_messages_interface' extension
    creating build
    creating build/temp.macosx-10.9-intel-2.7
    creating build/temp.macosx-10.9-intel-2.7/internals
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -Ideps/Eigen3/ -I/Users/yarden/Software/pyenv/lib/python2.7/site-packages/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.9-intel-2.7/internals/hmm_messages_interface.o -Ofast -std=c++11 -DHMM_TEMPS_ON_HEAP -DNDEBUG -w
    internals/hmm_messages_interface.cpp:10976:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:11254:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:11532:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:11810:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12088:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12366:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12644:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:12922:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:13200:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:13478:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:13756:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14034:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14312:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14590:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:14868:604: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:15146:612: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_12, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:269:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:44750:385: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:44993:389: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    internals/hmm_messages_interface.cpp:45236:385: error: cannot initialize a parameter of type 'int *' with an rvalue of type
          '__pyx_t_5numpy_int32_t *' (aka 'long *')
      ...(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_int32_t *, __pyx_pybuffernd_stateseq.rcbuffer->pybuffer.buf, __pyx_t_9, __pyx_pybuffernd_stateseq.diminfo[0].strides)))...
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    internals/hmm_messages.h:279:22: note: passing argument to parameter 'stateseq' here
                IntType *stateseq, FloatType *randseq)
                         ^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 errors generated.
    error: command 'cc' failed with exit status 1
    

    My version information:

    $ python --version
    Python 2.7.5
    $ python -c 'import numpy; print numpy.__version__'
    1.8.2
    

    Any thoughts on this? Thanks! Yarden

    opened by yarden 20
  • "Unexpected keyword argument" and "ndarray is not C contiguous"

    I'm trying to use this for some speaker diarization. I started getting some errors, I checked the examples to make sure it's not just my code. Here's a couple tracebacks:

    Traceback (most recent call last):
      File "concentration-resampling.py", line 45, in <module>
        posteriormodel.resample_model()
      File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 201, in resample_model
        self.resample_parameters(joblib_jobs=obs_jobs)
    TypeError: resample_parameters() got an unexpected keyword argument 'joblib_jobs'
    

    If I delete the parameter on that line, I get another error (included at the bottom because it's long). Since it was related to pybasicbayes I tried reverting to old commits, but none of them seem to work.

    Any ideas? I'm on Linux, as far as I know all my libraries are freshly installed and up to date.

    Traceback (most recent call last): File "concentration-resampling.py", line 45, in posteriormodel.resample_model() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 200, in resample_model self.resample_parameters() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 706, in resample_parameters super(_HSMMGibbsSampling,self).resample_parameters() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 205, in resample_parameters self.resample_trans_distn() File "/usr/local/lib/python2.7/dist-packages/pyhsmm/models.py", line 217, in resample_trans_distn self.trans_distn.resample([s.stateseq for s in self.states_list]) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/internals/transitions.py", line 374, in resample self._resample_beta(ms) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/internals/transitions.py", line 382, in _resample_beta self.beta_obj.resample(ms) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 2059, in resample self.alpha_0_obj.resample(counts) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 3225, in resample return super(GammaCompoundDirichlet,self).resample(data,niter=niter) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 3141, in resample a_n, b_n = self._posterior_hypparams(*self._get_statistics(data)) File "/usr/local/lib/python2.7/dist-packages/pyhsmm/basic/pybasicbayes/distributions.py", line 3237, in _get_statistics m = sample_crp_tablecounts(self.concentration,counts,self.weighted_cols) File "pyhsmm/basic/pybasicbayes/util/cstats.pyx", line 50, in pyhsmm.basic.pybasicbayes.util.cstats.sample_crp_tablecounts (basic/pybasicbayes/util/cstats.c:7911) File "pyhsmm/stringsource", line 614, in View.MemoryView.memoryview_cwrapper (basic/pybasicbayes/util/cstats.c:15841) File "pyhsmm/stringsource", line 321, in View.MemoryView.memoryview.cinit (basic/pybasicbayes/util/cstats.c:12387) ValueError: ndarray is not C-contiguous

    opened by UserAB1236872 20
  • Parallel resampling of observation distributions

    Parallel resampling of observation distributions

    Hi Matt, I have created a resample_model_parallel2 function which resamples the observation distributions in parrallel. This was at the cost of pushing the model into the engines an additional time. However, on my test this had very minor overload. I have added code to parallel.py .. that was easy!!! given the code you had in there already.

    In my initial test with Nmax = 100 the execution time came down to half :)

    i did not see a way to parallelise the sampling of the trans_dist but let me know if you think it is possible I can work on it. resampling of the initial state is super quick anyways so no point in speeding that up.

    Thanks a lot!

    Inti

    opened by inti 20
  • Failed to install pyhsmm on Mac

    Failed to install pyhsmm on Mac

    Below is my problem, I tried all the methods of installing pyhsmm, but still cannot install it on my Mac.

    danqing0703@Danqings-MacBook-Pro:~/pyhsmm/examples$ pip install pyhsmm Collecting pyhsmm Using cached pyhsmm-0.1.4.tar.gz Requirement already satisfied (use --upgrade to upgrade): Cython>=0.20.1 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): numpy in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): scipy in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): matplotlib in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): nose in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pybasicbayes>=0.1.3 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pyhsmm) Requirement already satisfied (use --upgrade to upgrade): six>=1.4 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pytz in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pyparsing>=1.5.6 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): mock in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): future in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from pybasicbayes>=0.1.3->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from mock->matplotlib->pyhsmm) Requirement already satisfied (use --upgrade to upgrade): funcsigs in /Users/danqing0703/anaconda/lib/python2.7/site-packages (from mock->matplotlib->pyhsmm) Building wheels for collected packages: pyhsmm Running setup.py bdist_wheel for pyhsmm Complete output from command /Users/danqing0703/anaconda/bin/python -c "import setuptools;file='/private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" bdist_wheel -d /var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/tmpsYliUapip-wheel-: running bdist_wheel running build running build_py creating build creating build/lib.macosx-10.5-x86_64-2.7 creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm copying pyhsmm/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm copying pyhsmm/models.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm copying pyhsmm/parallel.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/abstractions.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/distributions.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic copying pyhsmm/basic/models.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/basic creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/hmm_states.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/hsmm_inb_states.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/hsmm_states.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/initial_state.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals copying pyhsmm/internals/transitions.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/internals creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/plugins copying pyhsmm/plugins/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/plugins creating build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/init.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/general.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/plot.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/profiling.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/stats.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/testing.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util copying pyhsmm/util/text.py -> build/lib.macosx-10.5-x86_64-2.7/pyhsmm/util running build_ext building 'pyhsmm/internals.hmm_messages_interface' extension creating build/temp.macosx-10.5-x86_64-2.7 creating build/temp.macosx-10.5-x86_64-2.7/pyhsmm creating build/temp.macosx-10.5-x86_64-2.7/pyhsmm/internals gcc -fno-strict-aliasing -I/Users/danqing0703/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Ipyhsmm/deps/Eigen3 -I/Users/danqing0703/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/danqing0703/anaconda/include/python2.7 -c pyhsmm/internals/hmm_messages_interface.cpp -o build/temp.macosx-10.5-x86_64-2.7/pyhsmm/internals/hmm_messages_interface.o -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:276: pyhsmm/deps/Eigen3/Eigen/src/Core/MathFunctions.h:357:18: error: no member named 'log1p' in namespace 'std'; did you mean 'log10'? using std::log1p; ~~~~~^~~~~ log10 /usr/include/math.h:391:15: note: 'log10' declared here extern double log10(double); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:324: pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:302:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:303:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:304:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:373:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:374:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:439:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:440:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:329: pyhsmm/deps/Eigen3/Eigen/src/Core/PlainObjectBase.h:470:25: error: no member named 'move' in namespace 'std' : m_storage( std::move(other.m_storage) ) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:222:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:331: pyhsmm/deps/Eigen3/Eigen/src/Core/Array.h:134:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/internals/hmm_messages.h:224:18: note: in instantiation of template class 'Eigen::Matrix<int, -1, -1, 0, -1, -1>' requested here MatrixXi args(M,T); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::base; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:144:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::coeffRef; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:378:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::m_storage; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:73:20: note: in instantiation of template class 'Eigen::Matrix<float, -1, -1, 1, -1, -1>' requested here typedef typename PlainObjectType::Index Index; ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:32:31: note: in instantiation of template class 'Eigen::internal::traits<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here enum { has_direct_access = (traits::Flags & DirectAccessBit) ? 1 : 0, ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:109:32: note: in instantiation of template class 'Eigen::internal::accessors_level<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here int Level = internal::accessors_level::has_write_access ? WriteAccessors : ReadOnlyAccessors ^ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:105:12: note: in instantiation of default argument for 'MapBase<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' required here : public MapBase<Map<PlainObjectType, MapOptions, StrideType> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/internals/hmm_messages.h:31:24: note: in instantiation of template class 'Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> >' requested here NPMatrix eA(A,M,M); ^ pyhsmm/internals/hmm_messages.h:260:12: note: in instantiation of function template specialization 'hmm::messages_backwards_log' requested here { hmm::messages_backwards_log(M,T,A,aBl,betal); } ^ pyhsmm/internals/hmm_messages_interface.cpp:2961:15: note: in instantiation of member function 'hmmc<float, int>::messages_backwards_log' requested here __pyx_v_ref.messages_backwards_log((__pyx_v_A.shape[0]), (__pyx_v_aBl.shape[0]), (&(((float *) ( / dim=1 / ((char *) (((float *) ( / dim=0 / (__pyx_v_A.data + __pyx_t_1 * __pyx_v_A.strides[0]) )) + __pyx_t_2)) )))), (&(((float ) ( / dim=1 / ((char *) (((float *) ( / dim=0 _/ (__pyx_v_aBl.data + __pyx_t_3 * __pyx_v_aBl.strides[0]) )) + __pyx_t_4)) )))), (&(___Pyx_BufPtrCContig2d(float *, __pyx_pybuffernd_betal.rcbuffer->pybuffer.buf, __pyx_t_5, __pyx_pybuffernd_betal.diminfo[0].strides, __pyx_t_6, __pyx_pybuffernd_betal.diminfo[1].strides)))); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' using Base::base; ^~~~~~ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'gcc' failed with exit status 1


    Failed building wheel for pyhsmm Failed to build pyhsmm Installing collected packages: pyhsmm Running setup.py install for pyhsmm Complete output from command /Users/danqing0703/anaconda/bin/python -c "import setuptools, tokenize;file='/private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-5FdwS0-record/install-record.txt --single-version-externally-managed --compile: running install running build running build_py running build_ext building 'pyhsmm/internals.hmm_messages_interface' extension gcc -fno-strict-aliasing -I/Users/danqing0703/anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Ipyhsmm/deps/Eigen3 -I/Users/danqing0703/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/danqing0703/anaconda/include/python2.7 -c pyhsmm/internals/hmm_messages_interface.cpp -o build/temp.macosx-10.5-x86_64-2.7/pyhsmm/internals/hmm_messages_interface.o -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:276: pyhsmm/deps/Eigen3/Eigen/src/Core/MathFunctions.h:357:18: error: no member named 'log1p' in namespace 'std'; did you mean 'log10'? using std::log1p; ~~~~~^~~~~ log10 /usr/include/math.h:391:15: note: 'log10' declared here extern double log10(double); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:324: pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:302:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:303:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:304:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:373:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:374:21: error: no member named 'move' in namespace 'std' , m_cols(std::move(other.m_cols)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:439:21: error: no member named 'move' in namespace 'std' : m_data(std::move(other.m_data)) ~~~~~^ pyhsmm/deps/Eigen3/Eigen/src/Core/DenseStorage.h:440:21: error: no member named 'move' in namespace 'std' , m_rows(std::move(other.m_rows)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:329: pyhsmm/deps/Eigen3/Eigen/src/Core/PlainObjectBase.h:470:25: error: no member named 'move' in namespace 'std' : m_storage( std::move(other.m_storage) ) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:222:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:331: pyhsmm/deps/Eigen3/Eigen/src/Core/Array.h:134:19: error: no member named 'move' in namespace 'std' : Base(std::move(other)) ~~~~~^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/internals/hmm_messages.h:224:18: note: in instantiation of template class 'Eigen::Matrix<int, -1, -1, 0, -1, -1>' requested here MatrixXi args(M,T); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::base; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:144:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::coeffRef; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:378:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<int, -1, -1, 0, -1, -1>' using Base::m_storage; ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:403:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::derived;
    ^~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:73:20: note: in instantiation of template class 'Eigen::Matrix<float, -1, -1, 1, -1, -1>' requested here typedef typename PlainObjectType::Index Index; ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:32:31: note: in instantiation of template class 'Eigen::internal::traits<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here enum { has_direct_access = (traits::Flags & DirectAccessBit) ? 1 : 0, ^ pyhsmm/deps/Eigen3/Eigen/src/Core/util/ForwardDeclarations.h:109:32: note: in instantiation of template class 'Eigen::internal::accessors_level<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' requested here int Level = internal::accessors_level::has_write_access ? WriteAccessors : ReadOnlyAccessors ^ pyhsmm/deps/Eigen3/Eigen/src/Core/Map.h:105:12: note: in instantiation of default argument for 'MapBase<Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> > >' required here : public MapBase<Map<PlainObjectType, MapOptions, StrideType> > ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/internals/hmm_messages.h:31:24: note: in instantiation of template class 'Eigen::Map<Eigen::Matrix<float, -1, -1, 1, -1, -1>, 1, Eigen::Stride<0, 0> >' requested here NPMatrix eA(A,M,M); ^ pyhsmm/internals/hmm_messages.h:260:12: note: in instantiation of function template specialization 'hmm::messages_backwards_log' requested here { hmm::messages_backwards_log(M,T,A,aBl,betal); } ^ pyhsmm/internals/hmm_messages_interface.cpp:2961:15: note: in instantiation of member function 'hmmc<float, int>::messages_backwards_log' requested here __pyx_v_ref.messages_backwards_log((__pyx_v_A.shape[0]), (__pyx_v_aBl.shape[0]), (&(((float *) ( / dim=1 / ((char *) (((float *) ( / dim=0 / (__pyx_v_A.data + __pyx_t_1 * __pyx_v_A.strides[0]) )) + __pyx_t_2)) )))), (&(((float ) ( / dim=1 / ((char *) (((float *) ( / dim=0 _/ (__pyx_v_aBl.data + __pyx_t_3 * __pyx_v_aBl.strides[0]) )) + __pyx_t_4)) )))), (&(___Pyx_BufPtrCContig2d(float *, __pyx_pybuffernd_betal.rcbuffer->pybuffer.buf, __pyx_t_5, __pyx_pybuffernd_betal.diminfo[0].strides, __pyx_t_6, __pyx_pybuffernd_betal.diminfo[1].strides)))); ^ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:139:5: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' EIGEN_DENSE_PUBLIC_INTERFACE(Matrix) ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ pyhsmm/deps/Eigen3/Eigen/src/Core/util/Macros.h:404:9: note: expanded from macro 'EIGEN_DENSE_PUBLIC_INTERFACE' using Base::const_cast_derived; ^~~~~~ In file included from pyhsmm/internals/hmm_messages_interface.cpp:279: In file included from pyhsmm/internals/hmm_messages.h:4: In file included from pyhsmm/deps/Eigen3/Eigen/Core:330: pyhsmm/deps/Eigen3/Eigen/src/Core/Matrix.h:143:11: error: using declaration refers into 'Base::', which is not a base class of 'Matrix<float, -1, -1, 1, -1, -1>' using Base::base; ^~~~~~ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. error: command 'gcc' failed with exit status 1

    ----------------------------------------
    

    Command "/Users/danqing0703/anaconda/bin/python -c "import setuptools, tokenize;file='/private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-5FdwS0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/3h/t2d82x_163d9sh0rph_wgrw00000gn/T/pip-build-QaUnW3/pyhsmm

    opened by DanqingZ 18
  • Fixup for pypi distribution of pyhsmm

    Fixup for pypi distribution of pyhsmm

    I missed one wrinkle with pypi distribution. The previous version didn't include the *.pyx files in the distribution, and so the package from pypi would install correctly but would not compile the Cython files.

    For now, I included the Cython files and associated headers through MANIFEST.in. This will make it so when you install the package from pypi, the installation procedure automatically compiles the Cython files. A more ideal solution would be to write a conditional in the setup.py that checks if Cython is present, and if so, compiles the Cython files. For developers who want to make a distribution with python setup.py sdist, only the Cython-generated *.c files would go in the distribution, and then users could install with just Python and a C compiler, without needing Cython. (This more ideal setup is described here: http://stackoverflow.com/questions/4505747/how-should-i-structure-a-python-package-that-contains-cython-code). Since your users are mostly expert developers who probably have Cython, I figured it's not pressing.

    One caveat to watch out for: if you compile your pyx files to make *.c files, and have something that includes all *.c files in MANIFEST.in, you could accidentally package a Cython generated, e.g. foo.c along with a foo.pyx file. That could might create a situation where only the foo.c gets used even if it's out of date with the foo.pyx file. So to be on safe side, best to wipe out all Cython generated files before making a source distribution, i.e. before running python setup.py sdist.

    I fixed this in pybasicbayes too, see other pull request.

    It'd be great if you could make a fresh source distribution and upload it to pypi. You could wipe out the previous one through the pypi admin settings, or upgrade the VERSION variable in setup.py and make a new version release. I'm sure that if you wipe out the previous one and replace it without changing the version, you won't upset anyone, it has only been out for a few hours :)

    opened by yarden 17
  • pyhsmm compilation issue on windows

    pyhsmm compilation issue on windows

    Hi Guys,

    I am trying to compile pyhsmm with windows I am getting error,

    Please guide me How to fix this.

    My Environment ::

    Gnu Python-2.7 and [ Microsoft Visual Studio 11.0 ] windows 7 ( 64-bit ) Operating system.

    C:\Users\localadmin\pyhsmm> C:\Python27\python.exe .\setup.py build_ext --inplace nning build_ext ilding 'internals.hmm_messages_interface' extension eating build eating build\temp.win32-2.7 eating build\temp.win32-2.7\Release eating build\temp.win32-2.7\Release\internals \Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ideps/Eigen3/ -I \Python27\lib\site-packages\numpy\core\include -IC:\Python27\include -IC:\Python27\PC /Tpinternals\hmm_messages_interf e.cpp /Fobuild\temp.win32-2.7\Release\internals\hmm_messages_interface.obj -Ofast -std=c++11 -DHMM_TEMPS_ON_HEAP -DNDE G -w : Command line warning D9025 : overriding '/Os' with '/Ot' : Command line warning D9025 : overriding '/W3' with '/w' : Command line warning D9002 : ignoring unknown option '-Of' : Command line warning D9002 : ignoring unknown option '-Oa' : Command line warning D9002 : ignoring unknown option '-std=c++11' m_messages_interface.cpp \python27\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumP API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION \Program Files (x86)\Microsoft Visual Studio 11.0\VC\INCLUDE\xlocale(336) : warning C4530: C++ exception handler used, ut unwind semantics are not enabled. Specify /EHsc \users\localadmin\pyhsmm\internals\nptypes.h(24) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(24) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(30) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(30) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(36) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(36) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(61) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(61) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(67) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(67) : error C2059: syntax error : 'using' \users\localadmin\pyhsmm\internals\nptypes.h(73) : error C2988: unrecognizable template declaration/definition \users\localadmin\pyhsmm\internals\nptypes.h(73) : error C2059: syntax error : 'using' ternals\hmm_messages_interface.cpp(5120) : error C2664: 'hmmc::sample_forwards_log' : cannot convert parame r 7 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=float ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(5419) : error C2664: 'hmmc::sample_forwards_log' : cannot convert parame r 7 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=double ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(8753) : error C2664: 'hmmc::sample_backwards_normalized' : cannot conver parameter 5 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=float ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(9017) : error C2664: 'hmmc::sample_backwards_normalized' : cannot conver parameter 5 from '__pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=double ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(12382) : error C2664: 'hmmc::viterbi' : cannot convert parameter 6 from _pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=float ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ternals\hmm_messages_interface.cpp(12578) : error C2664: 'hmmc::viterbi' : cannot convert parameter 6 from _pyx_t_5numpy_int32_t *' to 'int32_t *' with [ FloatType=double ] Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast ror: command '"C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\BIN\cl.exe"' failed with exit status 2 C:\Users\localadmin\pyhsmm>

    opened by dhanasekaran-anbalagan 15
  • Can't compile on OS X 10.9.3

    Can't compile on OS X 10.9.3

    Hi,

    I'm running into a similar issue as https://github.com/mattjj/pyhsmm/issues/21. I'm on OS X 10.9.3, and I'm using clang (Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)).

    I tried building in two ways specified by the README but both failed for me.

    First I tried:

    $ python setup.py build_ext --inplace running build_ext gcc-4.2 not found, using clang instead building 'internals.hmm_messages_interface' extension clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP In file included from internals/hmm_messages_interface.cpp:360: In file included from internals/hmm_messages.h:4: In file included from deps/Eigen3/Eigen/Core:287: deps/Eigen3/Eigen/src/Core/MathFunctions.h:357:18: error: no member named 'log1p' in namespace 'std'; did you mean 'log10'? using std::log1p; ~~~~~^~~~~ log10 ... bunch of other errors follow ...

    I also tried the suggested alternative for clang but got: $ python setup.py build_ext --inplace --with-old-clang running build_ext gcc-4.2 not found, using clang instead building 'internals.hmm_messages_interface' extension clang -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -O2 -DNDEBUG -g -O3 -Ideps/Eigen3/ -I/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include -I/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c internals/hmm_messages_interface.cpp -o build/temp.macosx-10.6-intel-2.7/internals/hmm_messages_interface.o -O3 -w -DNDEBUG -std=c++11 -DHMM_TEMPS_ON_HEAP -stdlib=libc++ clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) error: command 'clang' failed with exit status 1

    Any ideas?

    Thanks

    opened by houbysoft 11
  • error with discrete observations in pyhsmm

    error with discrete observations in pyhsmm

    I'm trying to run pyhsmm on discrete (categorical) observations. Sampling runs, but using the sampled models gives errors. Code:

    Nmax = 25
    # observations
    obs_hypparams = {"K": 2,
                     "alpha_0": 1.}
    # gamma prior on duration 
    dur_hypparams = {'alpha_0': 1.,
                     'beta_0': 1.}
    obs_distns = [distributions.Categorical(**obs_hypparams) for state in range(Nmax)]
    dur_distns = [distributions.PoissonDuration(**dur_hypparams) for state in range(Nmax)]
    
    posteriormodel = pyhsmm.models.WeakLimitHDPHSMM(
            alpha=6.,gamma=6.,
            init_state_concentration=6.,
            obs_distns=obs_distns,
            dur_distns=dur_distns)
    
    # data
    data = np.array([0, 1] * 10 + [0, 0] * 10)
    posteriormodel.add_data(data)
    
    # inference
    models = []
    for idx in progprint_xrange(150):
        posteriormodel.resample_model()
        if (idx+1) % 10 == 0:
            models.append(copy.deepcopy(posteriormodel))
    
    # try to plot model (this fails)
    fig = plt.figure()
    for idx, model in enumerate(models):
        plt.clf()
        model.plot()
        plt.gcf().suptitle('HDP-HSMM sampled after %d iterations' % (10*(idx+1)))
    
    # try to predict (this fails)
    m = models[0]
    m.predict(np.array([]), 5)
    

    Prediction (m.predict) fails with the error:

        111     def predict(self,seed_data,timesteps,**kwargs):
    --> 112         full_data = np.vstack((seed_data,np.nan*np.ones((timesteps,seed_data.shape[1]))))
        113         self.add_data(full_data,**kwargs)
        114         s = self.states_list.pop()
    
    IndexError: tuple index out of range
    

    and plotting fails with:

    models.pyc in _plot_2d_data_scatter(self, ax, state_colors, plot_slice, update)
        308                 s._data_scatter.set_color(colorseq)
        309             else:
    --> 310                 s._data_scatter = ax.scatter(data[:,0],data[:,1],c=colorseq,s=5)
        311             artists.append(s._data_scatter)
        312 
    
    IndexError: too many indices for array
    

    I don't care about using plotting; just using it to test if model worked. I mainly want to call predict on the sampled models. Any thoughts on how to run pyhsmm with discrete observations? Thanks!

    question 
    opened by yarden 9
  • CVE-2007-4559 Patch

    CVE-2007-4559 Patch

    Patching CVE-2007-4559

    Hi, we are security researchers from the Advanced Research Center at Trellix. We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this blog.

    If you have further questions you may contact us through this projects lead researcher Kasimir Schulz.

    opened by TrellixVulnTeam 0
  • Energy disaggregation

    Energy disaggregation

    Hi,

    I would like to use your method for unsupervised energy disaggregation. The code base is pretty large and I struggle a little to really understand everything. Do you maybe still have the code to reproduce the disaggregation example from https://arxiv.org/pdf/1203.1365.pdf

    Best, Andreas

    opened by habring 0
  • Installing on linux (HPCC)

    Installing on linux (HPCC)

    I had some mild trouble installing pyhsmm on a linux-based computing cluster...documenting various errors and fixes in one place for anyone else who encounters them. Most have been solved in the issues before, but there are a few key points.

    1. You need to have the packages cython, requests, and futures installed before you run pyhsmm installation. (This is because the setup.py script imports them -- the script isn't smart enough to look in its own list of requirements before importing the modules it uses :).)
    2. Don't use python setup.py install, use pip install . This is critical. (See discussions e.g. here).
    3. You have to patch pybasicbayes manually (at least until the latest fix gets pushed to pypi)

    Full list of commands I ran: conda create -n pyhsmm_test_2 python=3.7 conda activate pyhsmm_test_2 pip install cython pip install requests pip install future cd [where you want the pyhsmm repo to live] git clone https://github.com/mattjj/pyhsmm.git cd pyhsmm pip install . pip install --upgrade git+https://github.com/mattjj/pybasicbayes.git

    And then double-check with: python /path/to/pyhsmm/examples/hmm.py

    opened by jonahpearl 0
  • Couldn't install pyhsmm on win 10: ERROR: Command errored out with exit status 1

    Couldn't install pyhsmm on win 10: ERROR: Command errored out with exit status 1

    Hi every one. I learned about this amazing hsmm library for python from a scientific paper. I now really need to use it. I tried to install it on my windows 10 laptop and pc. and none of them worked out. I searched every error I got and tried to solve them using previous issues.

    GCC

    I installed gcc using these recomendations. Then, this is my installed gcc version:

    gcc (MinGW.org GCC-6.3.0-1) 6.3.0
    Copyright (C) 2016 Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.  There is NO
    warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
    

    Microsoft Visual Studio 14.0

    I got errors regarding Ms VS14.0, so I downloaded ms build tools from Microsoft VisualStudio Downloads website and installed the MSVC v14.0 - VS 2015 C++ build tools (v14.00). I tried both enterprise and community version of vs installer.

    pyhsmm Installation

    then I tried installing the package using pip:

    pip install pyhsmm
    

    now here is what I get:

    (base) C:\Windows\system32>pip install pyhsmm
    Collecting pyhsmm
      Using cached pyhsmm-0.1.7.tar.gz (513 kB)
        ERROR: Command errored out with exit status 1:
         command: 'C:\ProgramData\Anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = 
    '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_322beb62dbe646499d9798cd3d759edd\\setup.py'"'"';
     __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
    ox7amkxz\\pyhsmm_322beb62dbe646499d9798cd3d759edd\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
    (__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 
    'C:\Users\USER\AppData\Local\Temp\pip-pip-egg-info-6ehmj4dx'
             cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_322beb62dbe646499d9798cd3d759edd\
        Complete output (26 lines):
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_322beb62dbe646499d9798cd3d759edd\setup.py", line
     76, in <module>
            urlretrieve(eigenurl, eigentarpath)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 247, in urlretrieve
            with contextlib.closing(urlopen(url, data)) as fp:
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 222, in urlopen
            return opener.open(url, data, timeout)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 525, in open
            response = self._open(req, data)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 542, in _open
            result = self._call_chain(self.handle_open, protocol, protocol +
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 502, in _call_chain
            result = func(*args)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1383, in http_open
            return self.do_open(http.client.HTTPConnection, req)
          File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1358, in do_open
            r = h.getresponse()
          File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1347, in getresponse
            response.begin()
          File "C:\ProgramData\Anaconda3\lib\http\client.py", line 307, in begin
            version, status, reason = self._read_status()
          File "C:\ProgramData\Anaconda3\lib\http\client.py", line 276, in _read_status
            raise RemoteDisconnected("Remote end closed connection without"
        http.client.RemoteDisconnected: Remote end closed connection without response
        Downloading Eigen...
        ----------------------------------------
    WARNING: Discarding https://files.pythonhosted.org/packages/2c/77/6d232566005a1ab9662f903e45122ce4e3679e881a8d0cb0d670c05a49bf/pyhsmm-0.1.7.tar.gz#sha256=3677225656bc39f7d9888361cfeba2c938437ba1df6c8d801985df040fb55f7a 
    (from https://pypi.org/simple/pyhsmm/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full 
    command output.
      Using cached pyhsmm-0.1.6.tar.gz (491 kB)
        ERROR: Command errored out with exit status 1:
         command: 'C:\ProgramData\Anaconda3\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = 
    '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_8899810deae1422dbe94c49cf35655af\\setup.py'"'"'; 
    __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
    ox7amkxz\\pyhsmm_8899810deae1422dbe94c49cf35655af\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
    (__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 
    'C:\Users\USER\AppData\Local\Temp\pip-pip-egg-info-88pfdwxz'
             cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_8899810deae1422dbe94c49cf35655af\
        Complete output (5 lines):
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_8899810deae1422dbe94c49cf35655af\setup.py", line 11, in <module>
            from urllib import urlretrieve
        ImportError: cannot import name 'urlretrieve' from 'urllib' (C:\ProgramData\Anaconda3\lib\urllib\__init__.py)
        ----------------------------------------
    WARNING: Discarding https://files.pythonhosted.org/packages/9d/91/9a6ea2ac2eb82953b253f3b16967a283ebda324923b4989c34df05de02f8/pyhsmm-0.1.6.tar.gz#sha256=1d000d709f9ba63fbb23e0d509f51dc41b5cc079253d6f4fcb7450a841764c3e (from https://pypi.org/simple/pyhsmm/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
      Using cached pyhsmm-0.1.4.tar.gz (2.0 MB)
    Requirement already satisfied: Cython>=0.20.1 in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (0.29.23)
    Requirement already satisfied: numpy in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.20.1)
    Requirement already satisfied: scipy in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.6.2)
    Requirement already satisfied: matplotlib in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (3.3.4)
    Requirement already satisfied: nose in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (1.3.7)
    Requirement already satisfied: pybasicbayes>=0.1.3 in c:\programdata\anaconda3\lib\site-packages (from pyhsmm) (0.2.2)
    Requirement already satisfied: future in c:\programdata\anaconda3\lib\site-packages (from pybasicbayes>=0.1.3->pyhsmm) (0.18.2)
    Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (2.4.7)
    Requirement already satisfied: python-dateutil>=2.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (2.8.1)
    Requirement already satisfied: kiwisolver>=1.0.1 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (1.3.1)
    Requirement already satisfied: pillow>=6.2.0 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (8.2.0)
    Requirement already satisfied: cycler>=0.10 in c:\programdata\anaconda3\lib\site-packages (from matplotlib->pyhsmm) (0.10.0)
    Requirement already satisfied: six in c:\programdata\anaconda3\lib\site-packages (from cycler>=0.10->matplotlib->pyhsmm) (1.15.0)
    Building wheels for collected packages: pyhsmm
      Building wheel for pyhsmm (setup.py) ... error
      ERROR: Command errored out with exit status 1:
       command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\USER\AppData\Local\Temp\pip-wheel-bkx1ousu'
           cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\
      Complete output (44 lines):
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-3.8
      creating build\lib.win-amd64-3.8\pyhsmm
      copying pyhsmm\models.py -> build\lib.win-amd64-3.8\pyhsmm
      copying pyhsmm\parallel.py -> build\lib.win-amd64-3.8\pyhsmm
      copying pyhsmm\__init__.py -> build\lib.win-amd64-3.8\pyhsmm
      creating build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\abstractions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\distributions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\models.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      copying pyhsmm\basic\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\basic
      creating build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\hmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\hsmm_inb_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\hsmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\initial_state.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\transitions.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      copying pyhsmm\internals\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\internals
      creating build\lib.win-amd64-3.8\pyhsmm\plugins
      copying pyhsmm\plugins\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\plugins
      creating build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\general.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\plot.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\profiling.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\stats.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\testing.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\text.py -> build\lib.win-amd64-3.8\pyhsmm\util
      copying pyhsmm\util\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\util
      running build_ext
      building 'pyhsmm\internals.hmm_messages_interface' extension
      creating build\temp.win-amd64-3.8
      creating build\temp.win-amd64-3.8\Release
      creating build\temp.win-amd64-3.8\Release\pyhsmm
      creating build\temp.win-amd64-3.8\Release\pyhsmm\internals
      C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ipyhsmm\deps\Eigen3 -IC:\ProgramData\Anaconda3\lib\site-packages\numpy\core\include -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /EHsc /Tppyhsmm\internals\hmm_messages_interface.cpp /Fobuild\temp.win-amd64-3.8\Release\pyhsmm\internals\hmm_messages_interface.obj -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP
      cl : Command line warning D9025 : overriding '/W3' with '/w'
      cl : Command line warning D9002 : ignoring unknown option '-O3'
      cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
      hmm_messages_interface.cpp
      C:\ProgramData\Anaconda3\include\pyconfig.h(205): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
      error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
      ----------------------------------------
      ERROR: Failed building wheel for pyhsmm
      Running setup.py clean for pyhsmm
    Failed to build pyhsmm
    Installing collected packages: pyhsmm
        Running setup.py install for pyhsmm ... error
        ERROR: Command errored out with exit status 1:
         command: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = 
    '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; 
    __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-
    ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)
    (__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 
    'C:\Users\USER\AppData\Local\Temp\pip-record-90tew8g2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\pyhsmm'
             cwd: C:\Users\USER\AppData\Local\Temp\pip-install-ox7amkxz\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\
        Complete output (44 lines):
        running install
        running build
        running build_py
        creating build
        creating build\lib.win-amd64-3.8
        creating build\lib.win-amd64-3.8\pyhsmm
        copying pyhsmm\models.py -> build\lib.win-amd64-3.8\pyhsmm
        copying pyhsmm\parallel.py -> build\lib.win-amd64-3.8\pyhsmm
        copying pyhsmm\__init__.py -> build\lib.win-amd64-3.8\pyhsmm
        creating build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\abstractions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\distributions.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\models.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        copying pyhsmm\basic\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\basic
        creating build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\hmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\hsmm_inb_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\hsmm_states.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\initial_state.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\transitions.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        copying pyhsmm\internals\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\internals
        creating build\lib.win-amd64-3.8\pyhsmm\plugins
        copying pyhsmm\plugins\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\plugins
        creating build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\general.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\plot.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\profiling.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\stats.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\testing.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\text.py -> build\lib.win-amd64-3.8\pyhsmm\util
        copying pyhsmm\util\__init__.py -> build\lib.win-amd64-3.8\pyhsmm\util
        running build_ext
        building 'pyhsmm\internals.hmm_messages_interface' extension
        creating build\temp.win-amd64-3.8
        creating build\temp.win-amd64-3.8\Release
        creating build\temp.win-amd64-3.8\Release\pyhsmm
        creating build\temp.win-amd64-3.8\Release\pyhsmm\internals
        C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -Ipyhsmm\deps\Eigen3 -IC:\ProgramData\Anaconda3\lib\site-packages\numpy\core\include -IC:\ProgramData\Anaconda3\include -IC:\ProgramData\Anaconda3\include "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE" "-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.10240.0\ucrt" /EHsc /Tppyhsmm\internals\hmm_messages_interface.cpp /Fobuild\temp.win-amd64-3.8\Release\pyhsmm\internals\hmm_messages_interface.obj -O3 -std=c++11 -DNDEBUG -w -DHMM_TEMPS_ON_HEAP
        cl : Command line warning D9025 : overriding '/W3' with '/w'
        cl : Command line warning D9002 : ignoring unknown option '-O3'
        cl : Command line warning D9002 : ignoring unknown option '-std=c++11'
        hmm_messages_interface.cpp
        C:\ProgramData\Anaconda3\include\pyconfig.h(205): fatal error C1083: Cannot open include file: 'basetsd.h': No such file or directory
        error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\x86_amd64\\cl.exe' failed with exit status 2
        ----------------------------------------
    ERROR: Command errored out with exit status 1: 'C:\ProgramData\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"'; __file__='"'"'C:\\Users\\USER\\AppData\\Local\\Temp\\pip-install-ox7amkxz\\pyhsmm_c3eec9d767c041c0aae0bbde31f7b514\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\USER\AppData\Local\Temp\pip-record-90tew8g2\install-record.txt' --single-version-externally-managed --compile --install-headers 'C:\ProgramData\Anaconda3\Include\pyhsmm' Check the logs for full command output.
    

    any help is much appreciated.

    opened by sinaeslami 1
  • HTTP 404 error when installing in WSL Debian

    HTTP 404 error when installing in WSL Debian

    OS: Windows Subsystem for Linux, Debian Miniconda3 command: pip install pyhsmm Here is the error message: Collecting pyhsmm Using cached pyhsmm-0.1.7.tar.gz (513 kB) ERROR: Command errored out with exit status 1: command: /home/whr/miniconda3/envs/somoseq/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/setup.py'"'"'; file='"'"'/tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vxgwk3u3 cwd: /tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/ Complete output (30 lines): Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-gnz7uh9d/pyhsmm_0402c4436f404942b8be4c3b7705f5bc/setup.py", line 76, in urlretrieve(eigenurl, eigentarpath) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 248, in urlretrieve with contextlib.closing(urlopen(url, data)) as fp: File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 223, in urlopen return opener.open(url, data, timeout) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 564, in error result = self._call_chain(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 756, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 570, in error return self._call_chain(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(*args) File "/home/whr/miniconda3/envs/somoseq/lib/python3.6/urllib/request.py", line 650, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 404: Not Found Downloading Eigen... ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

    It turns out that the link used to downloading "Eigen" package is not valid anymore.

    opened by FancyBrush 2
Owner
Matthew Johnson
research scientist @ Google Brain
Matthew Johnson
Uses MIT/MEDSL, New York Times, and US Census datasources to analyze per-county COVID-19 deaths.

Covid County Executive summary Setup Install miniconda, then in the command line, run conda create -n covid-county conda activate covid-county conda i

Ahmed Fasih 1 Dec 22, 2021
Probabilistic Programming in Python: Bayesian Modeling and Probabilistic Machine Learning with Theano

PyMC3 is a Python package for Bayesian statistical modeling and Probabilistic Machine Learning focusing on advanced Markov chain Monte Carlo (MCMC) an

PyMC 7.2k Dec 30, 2022
A Python package for Bayesian forecasting with object-oriented design and probabilistic models under the hood.

Disclaimer This project is stable and being incubated for long-term support. It may contain new experimental code, for which APIs are subject to chang

Uber Open Source 1.6k Dec 29, 2022
Statistical Rethinking: A Bayesian Course Using CmdStanPy and Plotnine

Statistical Rethinking: A Bayesian Course Using CmdStanPy and Plotnine Intro This repo contains the python/stan version of the Statistical Rethinking

Andrés Suárez 3 Nov 8, 2022
BAyesian Model-Building Interface (Bambi) in Python.

Bambi BAyesian Model-Building Interface in Python Overview Bambi is a high-level Bayesian model-building interface written in Python. It's built on to

null 861 Dec 29, 2022
Using approximate bayesian posteriors in deep nets for active learning

Bayesian Active Learning (BaaL) BaaL is an active learning library developed at ElementAI. This repository contains techniques and reusable components

ElementAI 687 Dec 25, 2022
A probabilistic programming library for Bayesian deep learning, generative models, based on Tensorflow

ZhuSuan is a Python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and

Tsinghua Machine Learning Group 2.2k Dec 28, 2022
Randomisation-based inference in Python based on data resampling and permutation.

Randomisation-based inference in Python based on data resampling and permutation.

null 67 Dec 27, 2022
Validation and inference over LinkML instance data using souffle

Translates LinkML schemas into Datalog programs and executes them using Souffle, enabling advanced validation and inference over instance data

Linked data Modeling Language 7 Aug 7, 2022
A probabilistic programming language in TensorFlow. Deep generative models, variational inference.

Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for fast experimentation and research with probabilis

Blei Lab 4.7k Jan 9, 2023
A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models, using the tools and APIs you know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

This tutorial's purpose is to introduce Pythonistas to methods for scaling their data science and machine learning work to larger datasets and larger models, using the tools and APIs they know and love from the PyData stack (such as numpy, pandas, and scikit-learn).

Coiled 102 Nov 10, 2022
🧪 Panel-Chemistry - exploratory data analysis and build powerful data and viz tools within the domain of Chemistry using Python and HoloViz Panel.

???? ??. The purpose of the panel-chemistry project is to make it really easy for you to do DATA ANALYSIS and build powerful DATA AND VIZ APPLICATIONS within the domain of Chemistry using using Python and HoloViz Panel.

Marc Skov Madsen 97 Dec 8, 2022
Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database.

Created covid data pipeline using PySpark and MySQL that collected data stream from API and do some processing and store it into MYSQL database.

null 2 Nov 20, 2021
Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

AWS Data Wrangler Pandas on AWS Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretMana

Amazon Web Services - Labs 3.3k Jan 4, 2023
First and foremost, we want dbt documentation to retain a DRY principle. Every time we repeat ourselves, we waste our time. Second, we want to understand column level lineage and automate impact analysis.

dbt-osmosis First and foremost, we want dbt documentation to retain a DRY principle. Every time we repeat ourselves, we waste our time. Second, we wan

Alexander Butler 150 Jan 6, 2023
Statistical Analysis 📈 focused on statistical analysis and exploration used on various data sets for personal and professional projects.

Statistical Analysis ?? This repository focuses on statistical analysis and the exploration used on various data sets for personal and professional pr

Andy Pham 1 Sep 3, 2022
Spectacular AI SDK fuses data from cameras and IMU sensors and outputs an accurate 6-degree-of-freedom pose of a device.

Spectacular AI SDK examples Spectacular AI SDK fuses data from cameras and IMU sensors (accelerometer and gyroscope) and outputs an accurate 6-degree-

Spectacular AI 94 Jan 4, 2023
Working Time Statistics of working hours and working conditions by industry and company

Working Time Statistics of working hours and working conditions by industry and company

Feng Ruohang 88 Nov 4, 2022