This PR includes a first draft of a .md file for chapter 1 and 2. Also includes other files like a bibtext (with all citations in the book). Figures for chapter 1 and 2. And a couple of editions to config files.

I think the .md is functional in the sense that equations are numbered, figures has captions, elements are crossreferenced (except for links to chapter still not available), the coolboxes are styled and code highlighted (although the style is not the same as the one used in the book), Tables are properly rendeted. Probably still room for improvements.

I was not able to automatically crossreference code blocks, so instead of consecutive numbers (as in the book) I am using ad hoc text references. Tables needs to be rewritten.

Manuscript code requires bambi package, please add to the environment.yml file.

Description

bambi library mentioned on page xix, Code 0.2 section, but not included in the environment.yml file so running the code on page xix yields a runtime error.

Reference page xix

Hi there, I'm encountering an issue in Chapter 06 at Code Block 6.14, when it comes to the section of the code where we're using generate_gam_ar_latent():

mcmc_samples, sampler_stats = run_mcmc(
    1000, gam_with_latent_ar, n_chains=4, num_adaptation_steps=1000,
    seed=tf.constant([36245, 734565], dtype=tf.int32),
    observed=co2_by_month_training_data.T
)

When it comes to running MCMC function on this model I get the following error:

------------------------------------------------------------------
ValueError                       Traceback (most recent call last)
<timed exec> in <module>

~\anaconda3\envs\pm3env\lib\site-packages\tensorflow\python\util\traceback_utils.py in error_handler(*args, **kwargs)
    151     except Exception as e:
    152       filtered_tb = _process_traceback_frames(e.__traceback__)
--> 153       raise e.with_traceback(filtered_tb) from None
    154     finally:
    155       del filtered_tb

~\anaconda3\envs\pm3env\lib\site-packages\tensorflow_probability\python\experimental\mcmc\windowed_sampling.py in windowed_adaptive_nuts(n_draws, joint_dist, n_chains, num_adaptation_steps, current_state, init_step_size, dual_averaging_kwargs, max_tree_depth, max_energy_diff, unrolled_leapfrog_steps, parallel_iterations, trace_fn, return_final_kernel_results, discard_tuning, chain_axis_names, seed, **pins)
    709       'unrolled_leapfrog_steps': unrolled_leapfrog_steps,
    710       'parallel_iterations': parallel_iterations}
--> 711   return _windowed_adaptive_impl(
    712       n_draws=n_draws,
    713       joint_dist=joint_dist,

~\anaconda3\envs\pm3env\lib\site-packages\tensorflow_probability\python\experimental\mcmc\windowed_sampling.py in _windowed_adaptive_impl(n_draws, joint_dist, kind, n_chains, proposal_kernel_kwargs, num_adaptation_steps, current_state, dual_averaging_kwargs, trace_fn, return_final_kernel_results, discard_tuning, seed, chain_axis_names, **pins)
    883       samplers.sanitize_seed(seed), n=2)
    884   (target_log_prob_fn, initial_transformed_position, bijector,
--> 885    step_broadcast, batch_shape, shard_axis_names) = _setup_mcmc(
    886        joint_dist,
    887        n_chains=n_chains,

~\anaconda3\envs\pm3env\lib\site-packages\tensorflow_probability\python\experimental\mcmc\windowed_sampling.py in _setup_mcmc(model, n_chains, init_position, seed, **pins)
    241                                    tf.nest.flatten(batch_shape))
    242 
--> 243   lp_static_shape = tensorshape_util.concatenate(n_chains, batch_shape)
    244 
    245   if not tensorshape_util.is_fully_defined(batch_shape):

~\anaconda3\envs\pm3env\lib\site-packages\tensorflow_probability\python\internal\tensorshape_util.py in concatenate(x, other)
    131       dimensions in `x` and `other`.
    132   """
--> 133   return _cast_tensorshape(tf.TensorShape(x).concatenate(other), type(x))
    134 
    135 

~\anaconda3\envs\pm3env\lib\site-packages\tensorflow_probability\python\internal\tensorshape_util.py in _cast_tensorshape(x, x_type)
     72     # as the shape, which we don't want.
     73     return np.array(as_list(x), dtype=np.int32)
---> 74   return x_type(as_list(x))
     75 
     76 

~\anaconda3\envs\pm3env\lib\site-packages\tensorflow_probability\python\internal\tensorshape_util.py in as_list(x)
     62     ValueError: If `x` has unknown rank.
     63   """
---> 64   return tf.TensorShape(x).as_list()
     65 
     66 

ValueError: as_list() is not defined on an unknown TensorShape.

Is there something that I need to fix in the generate_gam_ar_latent() function itself?

def generate_gam_ar_latent(training=True):

    @tfd.JointDistributionCoroutine
    def gam_with_latent_ar():
        seasonality, trend, noise_sigma = yield from gam_trend_seasonality()
        
        # Latent AR(1)
        ar_sigma = yield root(tfd.HalfNormal(.1, name='ar_sigma'))
        rho = yield root(tfd.Uniform(-1., 1., name='rho'))
        
        def ar_func(y):
            loc = tf.concat([tf.zeros_like(y[..., :1]), y[..., :-1]],
                            axis=-1) * rho[..., None]
            return tfd.Independent(
                tfd.Normal(loc=loc, scale=ar_sigma[..., None]),
                reinterpreted_batch_ndims=1
            )
        
        temporal_error = yield tfd.Autoregressive(
            distribution_fn=ar_func,
            sample0=tf.zeros_like(trend),
            num_steps=trend.shape[-1],
            name='temporal_error'
        )

        # Linear prediction
        y_hat = seasonality + trend + temporal_error
        
        if training:
            y_hat = y_hat[..., :co2_by_month_training_data.shape[0]]

        # Likelihood
        observed = yield tfd.Independent(
            tfd.Normal(y_hat, noise_sigma[..., None]),
            reinterpreted_batch_ndims=1,
            name='observed'
        )

    return gam_with_latent_ar

I upgraded to TensorFlow Probability 0.15.0 and now my Kernel just dies; is there any possible known solution to this? Many thanks for this incredible book!

Solutions to selected problems of CH3 at the bottom of the notebook (top of the notebook are the codes from the original chapter, which contains some snippets useful to solving the solutions at the bottom).

Problem 3H17 is incomplete but left for reference in case it maybe of help to others (feel free to delete if you want to keep full solutions only).

Hey, thanks once again for the great book - I was just revisiting the section on the Tensorflow Probability implementation of SARIMAX in Chapter 6, I have a couple quick questions:

1. Is it possible to use / include a demo of how you'd make out of sample forecast predictions using the SARIMAX models? (Probably requires more effort, but similar to the forecast examples given using the STS models at the end?)

2. While it's stated that we can't combine the seasonal_order with the design_matrix - is it possible to model external regressors (e.g. holidays, covariates etc) in addition to using the fourier_seasonality() basis function, and just add a couple extra columns to the design matrix, or is something more clever / sophisticated required?

Many thanks again!

Zen of Python Footnote error

• [x] Changed manuscript
• [x] Changed code notebook reference
• [x] Added to Errata

New York times truncated sentence

• [x] Changed manuscript
• [x] Changed open access
• [x] Added to Errata

Misspelled name and uncapitalized plinko

• [x] Changed manuscript
• [x] Changed open access
• [x] Added to Errata

Bad grammar in Figure 9.17 caption title

• [x] Changed manuscript
• [x] Changed open access
• [x] Added to Errata

Missing vertical line in Figure 3.3 https://github.com/BayesianModelingandComputationInPython/BookCode_Edition1/issues/106

• [x] Changed manuscript
• [x] Changed code notebook reference
• [x] Added to Errata

Clean up equation and make code example consistent

• \text for text within equations
• \texttt to set the variable names in monospace as in the following code example
• use same names in code example as in equation (also fixes bug in code example)

In the unpooled estimate the mean of the $\sigma$ estimate for salads is 21.3, whereas in the hierarchical estimate the mean of the same parameter estimate is now 25.5, and has been “pulled” up by the means of the pizza and sandwiches category. Moreover, the estimates of the pizza and salad categories in the hierarchical category, while regressed towards the mean slightly, remain largely the same as the unpooled estimates.

In the Hierarchical models section, the text lists salad as a category notably influenced by the hyperprior, then immediately lists salads as a stable category lightly effected by the hyperprior. I think this is meant to be sandwhiches instead of salads—since it's much more stable.

Description

This phrase in the intro to Hierarchical Models is missing a subject:

https://github.com/BayesianModelingandComputationInPython/BookCode_Edition1/blob/1ae02289e4b9961ad56e06de7700b1a1119d1856/markdown/chp_04.md?plain=1#L976-L979

I'm assuming it's meant to be something like hyperparameter. eg

The partial refers to the idea that groups that do not share one fixed parameter, but share a hyperparameter which describes the distribution of for the parameters of the prior itself.

Reference

Author Section

Do not close until

• [ ] Added to Errata
• [ ] Fixed in Open Access
• [ ] Fixed in latex source

Once again, enjoying the book here, thanks!

Model model_hierarchical_salad_sales_predictions in listing 4.17 is

@tfd.JointDistributionCoroutine
def out_of_sample_prediction_model():
    model = yield root(non_centered_model)
    β = model.beta_offset * model.beta_sigma[..., None] + model.beta_mu[..., None]
    
    β_group = yield tfd.Normal(
        model.beta_mu, model.beta_sigma, name="group_beta_prediction")
    group_level_prediction = yield tfd.Normal(
        β_group * out_of_sample_customers,
        model.sigma_prior,
        name="group_level_prediction")
    for l in [2, 4]:
        yield tfd.Normal(
            tf.gather(β, l, axis=-1) * out_of_sample_customers,
            tf.gather(model.sigma, l, axis=-1),
            name=f"location_{l}_prediction")

In the distribution for group_level_prediction, model.sigma_prior is used for the standard deviation. However, model.sigma_prior is the distribution of the hyperprior, not the prior that goes into the model.

I don't understand this. Shouldn't the standard deviation in group_level_prediction be a half-normal distribution whose scaling parameter in turn is influenced by the hyperprior, like this (see s = ... as well as the line afterwards):

@tfd.JointDistributionCoroutine
def out_of_sample_prediction_model():
    model = yield root(non_centered_model)
    β = model.beta_offset * model.beta_sigma[..., None] + model.beta_mu[..., None]
    
    β_group = yield tfd.Normal(
        model.beta_mu, model.beta_sigma, name="group_beta_prediction")
    s = yield tfd.HalfNormal(model.sigma_prior, name="s")
    group_level_prediction = yield tfd.Normal(
        β_group * out_of_sample_customers,
        s,
        name="group_level_prediction")
    for l in [2, 4]:
        yield tfd.Normal(
            tf.gather(β, l, axis=-1) * out_of_sample_customers,
            tf.gather(model.sigma, l, axis=-1),
            name=f"location_{l}_prediction")

This adds a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Not sure this is the best option. Just following other CRC books.

Description

Marginalize should be marginalizing

Reference

Author Section

Do not close until

• [ ] Added to Errata
• [ ] Fixed in Open Access
• [ ] Fixed in latex source

Description

may many

Need to delete may

Reference

Author Section

Do not close until

• [ ] Added to Errata
• [ ] Fixed in Open Access
• [ ] Fixed in latex source

Description

In section 11.6.1, I think Eq. (11.58) is not correct. The "+" should be "-". And the following Eqs. (11.59) - (11.61) should be rectified. Thanks.

Reference

Section 11.6.1

Author Section

Do not close until

• [x] Added to Errata
• [ ] Fixed in Open Access
• [ ] Fixed in latex source

Description

The code in Listing 5.1 splines_patsy line 1 may not be consistent with figure 5.6. We generate 500 data points using "x = np.linspace(0., 1., 500)" in the code. However, there are 20 data points in figure 5.6.

Reference Link to page for chp5 Splines

Author Section

Do not close until

• [ ] Added to Errata
• [ ] Fixed in Open Access
• [ ] Fixed in latex source

Maybe there is an typo below Equation (5.2).

This is known as polynomial regression. At first it may seem that Expression (5.2) is representing a multiple linear regression of the covariates $ \bm X, \bm X^2, \ldots +\bm X^m $.

should be

This is known as polynomial regression. At first it may seem that Expression (5.2) is representing a multiple linear regression of the covariates $ \bm X, \bm X^2, \ldots , \bm X^m $