Gaussian processes in TensorFlow

Last update: Jan 6, 2023

Overview

Website | Documentation (release) | Documentation (develop) | Glossary

What does GPflow do?
Installation
Getting Started with GPflow 2.0
The GPflow Community
Version Compatibility
- TensorFlow 1.x and GPflow 1.x
Citing GPflow

What does GPflow do?

GPflow is a package for building Gaussian process models in Python. It implements modern Gaussian process inference for composable kernels and likelihoods.

GPflow 2.1 builds on TensorFlow 2.2+ and TensorFlow Probability for running computations, which allows fast execution on GPUs.

The online documentation (develop)/(master) contains more details.

Maintainers

It was originally created by James Hensman and Alexander G. de G. Matthews. It is now actively maintained by (in alphabetical order) Artem Artemev, Mark van der Wilk, ST John, and Vincent Dutordoir. GPflow would not be the same without the community. We are grateful to all contributors who have helped shape GPflow.

GPflow is an open source project. If you have relevant skills and are interested in contributing then please do contact us (see "The GPflow community" section below).

Installation

Requirements

GPflow depends on both TensorFlow (TF, version ≥ 2.2) and TensorFlow Probability (TFP, version ≥ 0.10.1). We support Python ≥ 3.6.

NOTE: TensorFlow Probability releases are tightly coupled to TensorFlow, e.g. TFP 0.12 requires TF>=2.4, TFP 0.11 requires TF>=2.3, and TFP 0.10 requires TF>=2.2. Unfortunately, this is not specified in TFP's dependencies. So if you already have an (older) version of TensorFlow installed, GPflow will pull in the latest TFP, which will be incompatible. If you get errors such as ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.4, you have to either upgrade TensorFlow (pip install -U tensorflow) or manually install an older version of the tensorflow_probability package.

Latest (stable) release from PyPI

pip install gpflow

Latest (bleeding-edge) source from GitHub

Be aware that the develop branch may change regularly, and new commits may break your code.

In a check-out of the develop branch of the GPflow GitHub repository, run

pip install -e .

Alternatively, you can install the latest GitHub develop version using pip:

pip install git+https://github.com/GPflow/GPflow.git@develop#egg=gpflow

This will automatically install all required dependencies.

Getting Started with GPflow 2.0

There is an "Intro to GPflow 2.0" Jupyter notebook; check it out for details. To convert your code from GPflow 1 check the GPflow 2 upgrade guide.

The GPflow Community

Getting help

Bugs, feature requests, pain points, annoying design quirks, etc: Please use GitHub issues to flag up bugs/issues/pain points, suggest new features, and discuss anything else related to the use of GPflow that in some sense involves changing the GPflow code itself. You can make use of the labels such as bug, discussion, feature, feedback, etc. We positively welcome comments or concerns about usability, and suggestions for changes at any level of design.

We aim to respond to issues promptly, but if you believe we may have forgotten about an issue, please feel free to add another comment to remind us.

"How-to-use" questions: Please use Stack Overflow (gpflow tag) to ask questions that relate to "how to use GPflow", i.e. questions of understanding rather than issues that require changing GPflow code. (If you are unsure where to ask, you are always welcome to open a GitHub issue; we may then ask you to move your question to Stack Overflow.)

Slack workspace

We have a public GPflow slack workspace. Please use this invite link if you'd like to join, whether to ask short informal questions or to be involved in the discussion and future development of GPflow.

Contributing

All constructive input is gratefully received. For more information, see the notes for contributors.

Projects using GPflow

Projects building on GPflow and demonstrating its usage are listed below. The following projects are based on the current GPflow 2.x release:

Project	Description
Trieste	Bayesian optimization with TensorFlow, with out-of-the-box support for GPflow (2.x) models.
VFF	Variational Fourier Features for Gaussian Processes (GPflow 2.x version)
BranchedGP	Gaussian processes with branching kernels.
VBPP	Implementation of "Variational Bayes for Point Processes".
Gaussian Process Regression on Molecules	GPs to predict molecular properties by creating a custom-defined Tanimoto kernel to operate on Morgan fingerprints

If you would like your project listed here, let us know - or simply open a pull request that adds your project to the table above!

The following projects build on older versions of GPflow (pre-2020); we encourage their authors to upgrade to GPflow 2.

Project	Description
GPflowOpt	Bayesian Optimization using GPflow (stable release requires GPflow 0.5).
Doubly-Stochastic-DGP	Deep Gaussian Processes with Doubly Stochastic Variational Inference.
widedeepnetworks	Measuring the relationship between random wide deep neural networks and GPs.
orth_decoupled_var_gps	Variationally sparse GPs with orthogonally decoupled bases
kernel_learning	Implementation of "Differentiable Compositional Kernel Learning for Gaussian Processes".
DGPs_with_IWVI	Deep Gaussian Processes with Importance-Weighted Variational Inference
kerndisc	Library for automated kernel structure discovery in univariate data
Signature covariances	kernels for (time)series as inputs
Structured-DGP	Adding more structure to the variational posterior of the Doubly Stochastic Deep Gaussian Process

Version Compatibility

GPflow heavily depends on TensorFlow and as far as TensorFlow supports forward compatibility, GPflow should as well. The version of GPflow can give you a hint about backward compatibility. If the major version has changed then you need to check the release notes to find out how the API has been changed.

Unfortunately, there is no such thing as backward compatibility for GPflow models, which means that a model implementation can change without changing interfaces. In other words, the TensorFlow graph can be different for the same models from different versions of GPflow.

TensorFlow 1.x and GPflow 1.x

We have stopped development and support for GPflow based on TensorFlow 1. The latest release supporting TensorFlow 1 is v1.5.1. Documentation and tutorials will remain available.

Citing GPflow

To cite GPflow, please reference the JMLR paper. Sample Bibtex is given below:

@ARTICLE{GPflow2017,
  author = {Matthews, Alexander G. de G. and {van der Wilk}, Mark and Nickson, Tom and
	Fujii, Keisuke. and {Boukouvalas}, Alexis and {Le{\'o}n-Villagr{\'a}}, Pablo and
	Ghahramani, Zoubin and Hensman, James},
    title = "{{GP}flow: A {G}aussian process library using {T}ensor{F}low}",
  journal = {Journal of Machine Learning Research},
  year    = {2017},
  month = {apr},
  volume  = {18},
  number  = {40},
  pages   = {1-6},
  url     = {http://jmlr.org/papers/v18/16-537.html}
}

Since the publication of the GPflow paper, the software has been significantly extended with the framework for interdomain approximations and multioutput priors. We review the framework and describe the design in an arXiv paper, which can be cited by users.

@article{GPflow2020multioutput,
  author = {{van der Wilk}, Mark and Dutordoir, Vincent and John, ST and
            Artemev, Artem and Adam, Vincent and Hensman, James},
  title = {A Framework for Interdomain and Multioutput {G}aussian Processes},
  year = {2020},
  journal = {arXiv:2003.01115},
  url = {https://arxiv.org/abs/2003.01115}
}

Comments

Matrix Triangular Solve does not require CUSolver for GPU

I was just thinking about this on a walk. See section 3.4.6 of the linked document

https://developer.nvidia.com/sites/default/files/akamai/cuda/files/Misc/mygpu.pdf

Therefore our current bottleneck for SVGP on float32 can be moved to GPU without the difficult changes to the stream executor discussed here:

https://github.com/tensorflow/tensorflow/issues/2217

This is important because removing the MatrixTriangularSolve bottleneck would speed up GPU performance by about a factor of 3.

I guess CUSOLVER is only required for the Cholesky decomposition, which would be one of our next bottlenecks.

A look at Tom Nixon's fork confirms the GPU matrix triangular solve implementation only calls CUBLAS not CUSOLVER.

https://github.com/c0g/tensorflow/blob/master/tensorflow/core/kernels/matrix_triangular_solve_op.cc

I wish I had time to work on this right now but I don't unfortunately. Thought everyone should know straight away, anway.
discussion

opened by alexggmatthews 60
Kernel expectations
Ok, so again not ready to be merged, but a start to what we're discussing in #91. There are lots of things up for discussion and TODOs to merge the existing psi calculations in here. I'll try to summarise.

Notes on the commit here:

I have implemented exKxz which is the expectation <x_t K_{x_{t-1}, Z}>_q_{x_{t-1:t}}. This a necessary piece for GPSSMs, which I'm interested in. ~~It should collapse to eKxz for delta q(x_t).~~ (don't think this is true anymore).

The function exKxz computes the above expectation for an entire time series. So give it N+1 means (Xmu N+1 x D) and N+1 covariances and cross-covariances (Xcov 2xN+1xDxD). However, in order for this to work the cross covariance in Xcov[1, n-1, :, :] needs to be consistent with the covariance in Xcov[0, n, :, :]. This is ensured by theTriDiagonalBlockReptransform that I have added inetransforms.py`.

~~I haven't implemented eKxz yet.~~ This is now done.

~~I have Monte Carlo tests for all the expectations. However, the one for exKxz has high variance. I would appreciate help here.~~

I added TriDiagonalBlockRep along with the tests mostly because I wanted to upload something quickly. I feel like it may be a little outside of scope for GPflow, and it's not quite finished. Essentially it's a way to freely parameterise a covariance matrix with a Markovian structure (i.e. the precision is block tri diagonal). If this is something that could be useful / isn't minded in GPflow, I'll leave it in. Otherwise I'll rework test_ekern.py to not need it.

Overall things to do and decide:

~~Should we keep kernels with expectations separate from the original kernels? I.e. should the functionality in ekernels.py' be merged withkernels.py`?~~ No, keep them separate.

~~Add quadrature for 1D kernels in the kernel base class.~~

~~Add expectations to Linear kernel.~~

~~Sort out sum and product kernels.~~

~~Add a check that allows for diagonal q(X) as well.~~

~~Make tests more reliable. Perhaps we can do some low dimensional quadrature instead of relying on monte carlo? Currently variance causes things to fail sometimes.~~
opened by markvdw 46
Add MLP kernel

This pull request adds an implementation of the MLP kernel as used in GPy, together with a few tests. If there is interest to merge this PR, I can add more information to the docstring or maybe more tests. This code is also not tested very thoroughly yet.

opened by mrksr 29
Example of end-to-end training with GPFlow plus a neural net coded in tensorflow

Hello,

Is there any example on how to implement a neural network using tensorflow plus a final 'layer' implementing a GP process. Something similar to the architecture presented in: "Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks" here

I would like to be able to train the resulting model end-to-end. I suppose using the variational model it could be possible to optimize the GP parameters plus the NN weights using gradient descent, all at the same time, but I do not know how to code to connect the two models (tensorflow NN and GPflow GP)

Thank you

opened by mlopezm 28
GPflow has become considerably slower on GPU with some recent changes.
I have raised this as a bug because as you will see the effect is large.

The main changes I can see are:

The addition of user_ops for VecToTri and TriToVec

The use of batch matrix triangular solve.

The git hashes I'm comparing are:

new with changes 23b47f590806f0da3cb71916d68674f8bee4d875 old without changes 9c2faf62d04f8ec2ab35396553b5115d92d3c84b

I'm running MNIST training with a minibatch size of 1000.

GPU operation is 70% slower between these two hashes. CPU operation is about the same. Did we observe a speed up when we merged these?

The new profiling code #174 allows us to give a detailed breakdown of the slow down. user_ops_comparison.tar.gz

The can be viewed in chrome using the GUI found at chrome://tracing .

We find that the batch matrix triangular solve does not parallelize as well as the triangular_solve which is arguably a flaw in TensorFlow itself.

We find that the VecToTri and TriToVec ops are currently very slow relatively considering they just reshape something.

What should we do about this?

I would argue we should find out exactly which recent changes are causing this and revert them as soon as possible. The new code looks really great and requires considerably skill but this is showing us that it isn't ready for master yet IMHO. The new user ops also make installation more difficult. We want our users at any moment to have the best available experience and we have to be dedicated to this if we are going to achieve our ambitions for the software.

Longer term I argue that we should move the code to pull requests and expedite #174 #171 and #170

@markvdw @jameshensman thoughts?
bug
opened by alexggmatthews 28
serializing GPflow objects
This branch is perhaps mis-named, it does two things, both addressing #48

First, the ability to get all the parameters in a models in a dictionary.

Second, the ability to pickle GPflow objects.

This is very much up for discussion, and we may want to separate the two features. I think they're both useful though, and both address the same open question.

See also #56

import numpy as np import GPflow import pickle X, Y = np.random.randn(2, 10, 1) m = GPflow.sgpr.SGPR(X, Y, GPflow.kernels.Matern32(1), Z=X) s1 = pickle.dumps(m) m._compile() s2 = pickle.dumps(m) m1 = pickle.loads(s1) m2 = pickle.loads(s2) print m.compute_log_likelihood() print m1.compute_log_likelihood() print m2.compute_log_likelihood()
help wanted
opened by jameshensman 28
vector Gaussian likelihood variance for varying measurement uncertainty.

I would be very useful to have a vector variance in the Gaussian likelihood, as it is very often the case that measurements do not have a constant uncertainty.

opened by Joshuaalbert 26
Fix covariance overparameterisation

NB: This is nowhere near ready to commit yet. I just want some comments (from @alexggmatthews in particular).

This is a start to work to fix https://github.com/GPflow/GPflow/issues/49. This needs a new TensorFlow op to turn a vector into a triangular matrix. On the plus side, it's possible to do this without having to maintain a special fork of TensorFlow. With the TensorFlow binaries installed, custom ops can be compiled in place and then imported as a standard Python module.

The op is in /GPflow/tfops/vec_to_tri.cc, and needs to be compiled with the command: Linux: g++ -std=c++11 -shared zero_out.cc -o zero_out.so -fPIC -I $(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())') MacOS: g++ -undefined dynamic_lookup -std=c++11 -shared zero_out.cc -o zero_out.so -fPIC -I $(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')

There's a small example script calling the custom op in /op.py.

I've basically just adapted the TensorFlow custom op example, and need some pointers on how to proceed. @alexggmatthews, what resources did you use to find out the style used in TensorFlow and how to manipulate the Tensor objects etc? Right now I'm modifying individual elements in a very roundabout way, by first flattening them.

Next steps are to properly template the ops to ensure compatibility with more data types, adding gradients, tests etc...
enhancement please review and merge

opened by markvdw 24

Issues in GP regression with the Matern Kernel. Is RBF more numerically stable than the Matern Kernel?

I am doing GP regression with some real data dim(X)=[N,3] and dim[Y] = [N,1]. When I use the RBF Kernel it works fine when I use the Matern kernel it gives me some errors, probably in computing the cholesky during the optimazation. Here an idea of my code:

import GPflow

m = GPflow.gpr.GPR(X, Y, kern=GPflow.kernels.RBF(3))
m.optimize()


m2 = GPflow.gpr.GPR(X, Y, kern=GPflow.kernels.Matern52(3)+ GPflow.kernels.White(input_dim=3))
m2.optimize()

This is the error that I obtain

2017-08-21 17:49:05.142629: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-21 17:49:05.142639: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-21 17:49:05.142641: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-08-21 17:49:05.142643: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-08-21 17:49:05.142645: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
2017-08-21 17:49:05.246520: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2017-08-21 17:49:05.246789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties: 
name: GeForce GTX 1080
major: 6 minor: 1 memoryClockRate (GHz) 1.86
pciBusID 0000:01:00.0
Total memory: 7.92GiB
Free memory: 7.16GiB
2017-08-21 17:49:05.246798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 
2017-08-21 17:49:05.246801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y 
2017-08-21 17:49:05.246806: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
2017-08-21 17:49:06.786071: I tensorflow/core/kernels/cuda_solvers.cc:137] Creating CudaSolver handles for stream 0x510a320
2017-08-21 17:49:11.326938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
2017-08-21 17:49:13.169500: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Got info = 49 for batch index 0, expected info = 0. Debug_info =potrf
Traceback (most recent call last):
  File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1596, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1023, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/homes/romeres/Projects/1DModelCodes/syncinputs_v2.py", line 177, in <module>
    m2.optimize()
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 223, in optimize
    return self._optimize_np(method, tol, callback, maxiter, **kw)
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 308, in _optimize_np
    options=options)
  File "/usr/lib/python2.7/dist-packages/scipy/optimize/_minimize.py", line 447, in minimize
    callback=callback, **options)
  File "/usr/lib/python2.7/dist-packages/scipy/optimize/lbfgsb.py", line 330, in _minimize_lbfgsb
    f, g = func_and_grad(x)
  File "/usr/lib/python2.7/dist-packages/scipy/optimize/lbfgsb.py", line 278, in func_and_grad
    f = fun(x, *args)
  File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 289, in function_wrapper
    return function(*(wrapper_args + args))
  File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 63, in __call__
    fg = self.fun(x, *args)
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 43, in __call__
    f, g = self._objective(x)
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 161, in obj
    feed_dict=feed_dict)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
    run_metadata_ptr)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
    feed_dict_string, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
    target_list, options, run_metadata)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Got info = 49 for batch index 0, expected info = 0. Debug_info =potrf
	 [[Node: name.build_likelihood/Cholesky = Cholesky[T=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/gpu:0"](name.build_likelihood/add)]]
	 [[Node: name.build_likelihood/Cholesky/_27 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_394_name.build_likelihood/Cholesky", tensor_type=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Caused by op u'name.build_likelihood/Cholesky', defined at:
  File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1596, in <module>
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1023, in run
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/homes/romeres/Projects/1DModelCodes/syncinputs_v2.py", line 177, in <module>
    m2.optimize()
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 223, in optimize
    return self._optimize_np(method, tol, callback, maxiter, **kw)
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 284, in _optimize_np
    self._compile()
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 133, in _compile
    f = self.build_likelihood() + self.build_prior()
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/scoping.py", line 43, in runnable
    return f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/gpr.py", line 60, in build_likelihood
    L = tf.cholesky(K)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_linalg_ops.py", line 227, in cholesky
    result = _op_def_lib.apply_op("Cholesky", input=input, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Got info = 49 for batch index 0, expected info = 0. Debug_info =potrf
	 [[Node: name.build_likelihood/Cholesky = Cholesky[T=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/gpu:0"](name.build_likelihood/add)]]
	 [[Node: name.build_likelihood/Cholesky/_27 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_394_name.build_likelihood/Cholesky", tensor_type=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]


Process finished with exit code 1

I obtained this error from the PyCharm console but from terminal it is analogous. Line 177 corresponds to m2.optimize().

As you can see I tried to add White noise to overcome possible numerical error. I also tried to constrain the hyperparameters within a certain range but the result did not change.

m2.kern.matern52.lengthscales.transform = GPflow.transforms.Logistic(0.1, 10.)
m2.kern.matern52.variance.transform = GPflow.transforms.Logistic(0.1, 10.)

Am I missing something or is there a problem in the Matern kernel implementation? Thank you in advance.

opened by DiegoRomeres 23

GPflow 2.0
Hello Everyone!

TensorFlow 2.0 is coming and will bring important changes. In short, TensorFlow will have function style execution by making Eager mode default, details are here. This is very exciting news, as with "dynamic-like" graph creation users will have more flexibility and be able to do more (e.g. proper for loops and conditions). I started awav/gpflow-2.0 branch as an experimental solution and eager mode playground, but eventually awav/gpflow-2.0 will be merged into the GPflow master. GPflow 2.0 branch is expected to be unstable for now.

Any feedback and suggestions are welcome!

Plans of what we would like to do:

At first, integrate GPflow with tensorflow_probabilities.

We are considering to support both current TensorFlow static graph execution and Eager modes. It is reasonable, but implies difficulties in implementation.

Unify saver interface with possibility to use TensorFlow (Keras) saver.

Support Keras monitoring as well.

What awav/gpflow-2.0 provides now:

Project layout is cleaned and "optimized".

Autoflow methods are removed, there is no build_* functions.

@params_as_tensors are removed. There is no more such things as tf.Tensor which you can run via session.run and Parameter class is actually inherited from Eager Variable.

TensorFlow Bijectors are used as Parameter transforms.

TensorFlow distributions are used as Parameter priors.

Parameterized is renamed to Module

There is a prototype example. For now only GPR and SVGP models are ported. In that file you will find rough example how to use SVGP and what you can do with GPflow 2.0.

Thank you!
discussion important announcement
opened by awav 22
Tracer and wrapped session for GPflow
Not to be merged yet.

This PR currently allows one to switch on a profiling mode that stores a tracer file for calls to tf.session.run . It can be set to store a tracer file for each such call or only the last one.

To do this there are two key concepts:

TensorFlow itself has the ability to do this either for both CPU and GPU processes. They key commands are

run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE) run_metadata = tf.RunMetadata() sess.run(res, options=run_options, run_metadata=run_metadata) tl = timeline.Timeline(run_metadata.step_stats) ctf = tl.generate_chrome_trace_format() with open('timeline.json', 'w') as f: f.write(ctf)

TensorFlow (oddly) does not have the ability to do this every time with some constant run_options. Therefore I have wrapped all calls to session in GPflow. We can then use either a standard tf.Session or a special new class that has the same interface as the normal session but is running and storing the traces each time. I have done it this way to minimize the impact on the rest of the code.

I can imagine we might want other interfaces for the profiler which is why I'm doing the PR a bit early.

I still need to write unit tests and update the existing ones.

An example script is provided in notebooks/example_profiler.py
enhancement help wanted
opened by alexggmatthews 21
Noted cpython 3.8 bug
PR type: doc improvement

Related issue(s)/PRs

Summary

The bug in cpython3.8 linked above is causing our benchmarking to fail. This isn't really fixable by us, so I've added a warning comment to prevent people banging their heads against this issue.

Fully backwards compatible: yes

PR checklist

[ ] New features: code is well-documented

[ ] detailed docstrings (API documentation)

[ ] notebook examples (usage demonstration)

[ ] The bug case / new feature is covered by unit tests

[ ] Code has type annotations

[x] Build checks

[x] I ran the black+isort formatter (make format)

[x] I locally tested that the tests pass (make check-all)

[ ] Release management

[ ] RELEASE.md updated with entry for this change

[ ] New contributors: I've added myself to CONTRIBUTORS.md
opened by sc336 1

Check shape warning when importing gpflow

Running import gpflow in the latest release outputs the following warnings:

In [1]: import gpflow
/home/uri.granta/.pyenv/versions/3.7.13/envs/trieste/lib/python3.7/site-packages/gpflow/experimental/utils.py:43: UserWarning: You're calling gpflow.experimental.check_shapes.decorator.check_shapes which is considered *experimental*. Expect: breaking changes, poor documentation, and bugs.
  f"You're calling {name} which is considered *experimental*."
/home/uri.granta/.pyenv/versions/3.7.13/envs/trieste/lib/python3.7/site-packages/gpflow/experimental/utils.py:43: UserWarning: You're calling gpflow.experimental.check_shapes.inheritance.inherit_check_shapes which is considered *experimental*. Expect: breaking changes, poor documentation, and bugs.
  f"You're calling {name} which is considered *experimental*."

While it may be reasonable to output a warning when a user uses check shape for the first time, we probably shouldn't output it when gpflow itself does.

bug

opened by uri-granta 1

Demonstrate best practices in the notebooks: `@tf.function`

Documentation/tutorial notebooks

Are there any mistakes in the docs? Some notebooks (e.g. https://gpflow.github.io/GPflow/develop/notebooks/advanced/natural_gradients.html, there may be more, I did not check extensively) do not use @tf.function when using a non-Scipy optimizer (our Scipy class automatically wraps the optimisation in tf.function by default). This is significantly slower in practice, and for any real data sets you would want to ensure the tf.function-precompilation. The notebooks should demonstrate this best practice, so users do not wonder why GPflow is so slow...

opened by st-- 0
Added _sparse_orthogonal_conditional
Draft PR for taking the GPflow components from this GPflux PR into GPflow. The work is based on this paper.

============================

PR type: bugfix / enhancement / new feature / doc improvement

Related issue(s)/PRs:

Summary

Proposed changes

...

...

...

What alternatives have you considered?

Minimal working example

# Put your example code in here

Release notes

Fully backwards compatible: yes / no

If not, why is it worth breaking backwards compatibility:

PR checklist

[ ] New features: code is well-documented

[ ] detailed docstrings (API documentation)

[ ] notebook examples (usage demonstration)

[ ] The bug case / new feature is covered by unit tests

[ ] Code has type annotations

[ ] Build checks

[ ] I ran the black+isort formatter (make format)

[ ] I locally tested that the tests pass (make check-all)

[ ] Release management

[ ] RELEASE.md updated with entry for this change

[ ] New contributors: I've added myself to CONTRIBUTORS.md
opened by sc336 1
turn jitter into an argument of inv_probit instead of hard-coded inside
Summary

Turn jitter of inv_probit utility function into an argument. Add docstring.

Release notes

Fully backwards compatible: yes

PR checklist

[x] New features: code is well-documented

[x] detailed docstrings (API documentation)

[ ] notebook examples (usage demonstration)

[ ] The bug case / new feature is covered by unit tests

[x] Code has type annotations

[ ] Build checks

[ ] I ran the black+isort formatter (make format)

[ ] I locally tested that the tests pass (make check-all)

[ ] Release management

[ ] RELEASE.md updated with entry for this change

[ ] New contributors: I've added myself to CONTRIBUTORS.md
opened by st-- 0

Releases(v2.6.4)

v2.6.4(Dec 2, 2022)
This is yet another bug-fix release.

Bug Fixes and Other Changes

Fix to to_default_float to avoid losing precision when called with python floats.

Thanks to our Contributors

This release contains contributions from:

ChrisMorter
Source code(tar.gz)
Source code(zip)
v2.6.3(Oct 13, 2022)
Release 2.6.3 (next upcoming release in progress)

This is yet another bug-fix release.

Bug Fixes and Other Changes

Fix to check_shapes handling of tfp..._TensorCoercible.

Thanks to our Contributors

This release contains contributions from:

jesnie
Source code(tar.gz)
Source code(zip)
v2.6.2(Oct 10, 2022)
Release 2.6.2

This is a bug-fix release, for compatibility with GPflux.

Bug Fixes and Other Changes

Extract shapes of tfp.python.layers.internal.distribution_tensor_coercible._TensorCoercible.

Allow FallbackSeparateIndependentInducingVariables to have children with different shapes.

Allow input and output batches on GaussianQuadrature to be different.

Thanks to our Contributors

This release contains contributions from:

jesnie
Source code(tar.gz)
Source code(zip)
v2.6.1(Sep 29, 2022)
Release 2.6.1

This is a bug-fixes release, due to problems with model saving in 2.6.0.

Breaking Changes

Removed gpflow.utilities.ops.cast. Use tf.cast instead.

Bug Fixes and Other Changes

Fixed bug related to tf.saved_model and methods wrapped in @check_shapes.

Some documentation formatting fixes.

Thanks to our Contributors

This release contains contributions from:

jesnie
Source code(tar.gz)
Source code(zip)
v2.6.0(Sep 20, 2022)
Release 2.6.0

The major theme for this release is heteroskedastic likelihoods. Changes have unfortunately caused some breaking changes, but makes it much easier to use heteroskedastic likelihoods, either by plugging together built-in GPflow classes, or when writing your own. See our updated notebook, for examples on how to use this.

Breaking Changes

All likelihood methods now take an extra X argument. If you have written custom likelihoods or you have custom code calling likelihoods directly you will need to add this extra argument.

On the CGLB model the xnew parameters has changed name to Xnew, to be consistent with the other models.

On the GPLVM model the variance returned by predict_f with full_cov=True has changed shape from [batch..., N, N, P] to [batch..., P, N, N] to be consistent with the other models.

gpflow.likelihoods.Gaussian.DEFAULT_VARIANCE_LOWER_BOUND has been replaced with gpflow.likelihoods.scalar_continuous.DEFAULT_LOWER_BOUND.

Change to InducingVariables API. InducingVariables must now have a shape property.

gpflow.experimental.check_shapes.get_shape.register has been replaced with gpflow.experimental.check_shapes.register_get_shape.

check_shapes will no longer automatically wrap shape checking in tf.compat.v1.flags.tf_decorator.make_decorator. This is likely to affect you if you use check_shapes with custom Keras models. If you require the decorator you can manually enable it with check_shapes(..., tf_decorator=True).

Known Caveats

Shape checking is now, by default, disabled within tf.function. Use set_enable_check_shapes to change this behaviour. See the API documentation for more details.

Major Features and Improvements

Improved handling of variable noise

All likelihood methods now take an X argument, allowing you to easily implement heteroskedastic likelihoods.

The Gaussian likelihood can now be parametrized by either a variance or a scale

Some existing likelihoods can now take a function (of X) instead of a parameter, allowing them to become heteroskedastic. The parameters are:

Gaussian variance

Gaussian scale

StudentT scale

Gamma shape

Beta scale

The GPR and SGPR can now be configured with a custom Gaussian likelihood, allowing you to make them heteroskedastic.

See the updated notebook.

gpflow.mean_functions has been renamed gpflow.functions, but with an alias, to avoid breaking changes.

gpflow.experimental.check_shapes

Can now be in three different states - ENABLED, EAGER_MODE_ONLY, and DISABLE. The default is EAGER_MODE_ONLY, which only performs shape checks when the code is not compiled. Compiling the shape checking code is a major bottleneck and this provides a significant speed-up for performance sensitive parts of the code.

Now supports multiple variable-rank dimensions at the same time, e.g. cov: [n..., n...].

Now supports single broadcast dimensions to have size 0 or 1, instead of only 1.

Now supports variable-rank dimensions to be broadcast, even if they're not leading.

Now supports is None and is not None as checks for conditional shapes.

Now uses custom function register_get_shape instead of get_shape.register, for better compatibility with TensorFlow.

Now supports checking the shapes of InducingVariables.

Now adds documentation to function arguments that has declared shapes, but no other documentation.

All of GPflow is now consistently shape-checked.

All built-in kernels now consistently support broadcasting.

Bug Fixes and Other Changes

Tested with TensorFlow 2.10.

Add support for Apple Silicon Macs (arm64) via the tensorflow-macos dependency. (#1850)

New implementation of GPR and SGPR posterior objects. This primarily improves numerical stability. (#1960)

For the GPR this is also a speed improvement when using a GPU.

For the SGPR this is a mixed bag, performance-wise.

Improved checking and error reporting for the models than do not support full_cov and full_output_cov.

Documentation improvements:

Improved MCMC notebook.

Deleted notebooks that had no contents.

Fixed some broken formatting.

Thanks to our Contributors

This release contains contributions from:

jesnie, corwinpro, st--, vdutor
Source code(tar.gz)
Source code(zip)
v2.5.2(May 10, 2022)
Release 2.5.2

This release fixes a performance regression introduced in 2.5.0. 2.5.0 used features of Python that tensorfow < 2.9.0 do not know how to compile, which negatively impacted performance.

Bug Fixes and Other Changes

Fixed some bugs that prevented TensorFlow compilation and had negative performance impact. (#1882)

Various improvements to documentation. (#1875, #1866, #1877, #1879)

Thanks to our Contributors

This release contains contributions from:

jesnie
Source code(tar.gz)
Source code(zip)
v2.5.1(Apr 28, 2022)
Release 2.5.1

Fix problem with release process of 2.5.0.

Bug Fixes and Other Changes

Fix bug in release process.

Thanks to our Contributors

This release contains contributions from:

jesnie
Source code(tar.gz)
Source code(zip)
v2.5.0(Apr 28, 2022)
Release 2.5.0

The focus of this release has mostly been bumping the minimally supported versions of Python and TensorFlow; and development of gpflow.experimental.check_shapes.

Breaking Changes

Dropped support for Python 3.6. New minimum version is 3.7. (#1803, #1859)

Dropped support for TensorFlow 2.2 and 2.3. New minimum version is 2.4. (#1803)

Removed sub-package gpflow.utilities.utilities. It was scheduled for deletion in 2.3.0. Use gpflow.utilities instead. (#1804)

Removed method Likelihood.predict_density, which has been deprecated since March 24, 2020. (#1804)

Removed property ScalarLikelihood.num_gauss_hermite_points, which has been deprecated since September 30, 2020. (#1804)

Known Caveats

Further improvements to type hints - this may reveal new problems in your code-base if you use a type checker, such as mypy. (#1795, #1799, #1802, #1812, #1814, #1816)

Major Features and Improvements

Significant work on gpflow.experimental.check_shapes.

Support anonymous dimensions. (#1796)

Add a hook to let the user register shapes for custom types. (#1798)

Support Optional values. (#1797)

Make it configurable. (#1810)

Add accesors for setting/getting previously applied checks. (#1815)

Much improved error messages. (#1822)

Add support for user notes on shapes. (#1836)

Support checking all elements of collections. (#1840)

Enable stand-alone shape checking, without using a decorator. (#1845)

Support for broadcasts. (#1849)

Add support for checking the shapes of intermediate computations. (#1853)

Support conditional shapes. (#1855)

Significant speed-up of the GPR posterior objects. (#1809, #1811)

Significant improvements to documentation. Note the new home page: https://gpflow.github.io/GPflow/index.html (#1828, #1829, #1830, #1831, #1833, #1841, #1842, #1856, #1857)

Bug Fixes and Other Changes

Minor improvement to code clarity (variable scoping) in SVGP model. (#1800)

Improving mathematical formatting in docs (SGPR derivations). (#1806)

Allow anisotropic kernels to have negative length-scales. (#1843)

Thanks to our Contributors

This release contains contributions from:

ltiao, uri.granta, frgsimpson, st--, jesnie
Source code(tar.gz)
Source code(zip)
v2.4.0(Mar 1, 2022)
Release 2.4.0

This release mostly focuses on make posterior objects useful for Bayesian Optimisation. It also adds a new experimetal sub-package, with a tool for annotating tensor shapes.

Breaking Changes

Slight change to the API of custom posterior objects. gpflow.posteriors.AbstractPosterior._precompute no longer must return an alpha and an Qinv - instead it returns any arbitrary tuple of PrecomputedValues. Correspondingly gpflow.posteriors.AbstractPosterior._conditional_with_precompute should no longer try to access self.alpha and self.Qinv, but instead is passed the tuple of tensors returned by _precompute, as a parameter. (#1763, #1767)

Slight change to the API of inducing points. You should no longer override gpflow.inducing_variables.InducingVariables.__len__. Override gpflow.inducing_variables.InducingVariables.num_inducing instead. num_inducing should return a tf.Tensor which is consistent with previous behaviour, although the type previously was annotated as int. __len__ has been deprecated. (#1766, #1792)

Known Caveats

Type hints have been added in several places - this may reveal new problems in your code-base if you use a type checker, such as mypy. (#1766, #1769, #1771, #1773, #1775, #1777, #1780, #1783, #1787, #1789)

Major Features and Improvements

Add new posterior class to enable faster predictions from the VGP model. (#1761)

VGP class bug-fixed to work with variable-sized data. Note you can use gpflow.models.vgp.update_vgp_data to ensure variational parameters are updated sanely. (#1774).

All posterior classes bug-fixed to work with variable data sizes, for Bayesian Optimisation. (#1767)

Added experimental sub-package for features that are still under developmet.

Added gpflow.experimental.check_shapes for checking tensor shapes. (#1760, #1768, #1782, #1785, #1788)

Bug Fixes and Other Changes

Make dataclasses dependency conditional at install time. (#1759)

Simplify calculations of some predict_f. (#1755)

Thanks to our Contributors

This release contains contributions from:

jesnie, tmct, joacorapela
Source code(tar.gz)
Source code(zip)
v2.3.1(Jan 20, 2022)
Release 2.3.1

This is a bug-fix release, primarily for the GPR posterior object.

Bug Fixes and Other Changes

GPR posterior

Fix the calculation in the GPR posterior object (#1734).

Fixes leading dimension issues with GPRPosterior._conditional_with_precompute() (#1747).

Make gpflow.optimizers.Scipy able to handle unused / unconnected variables. (#1745).

Build

Fixed broken CircleCi build (#1738).

Update CircleCi build to use next-gen Docker images (#1740).

Fixed broken triggering of docs generation (#1744).

Make all slow tests depend on fast tests (#1743).

Make make dev-install also install the test requirements (#1737).

Documentation

Fixed broken link in README.md (#1736).

Fix broken build of cglb.ipynb (#1742).

Add explanation of how to run notebooks locally (#1729).

Fix formatting in notebook on Heteroskedastic Likelihood (#1727).

Fix broken link in introduction (#1718).

Test suite

Amends test_gpr_posterior.py so it will cover leading dimension uses.

Thanks to our Contributors

This release contains contributions from:

st--, jesnie, johnamcleod, Andrew878
Source code(tar.gz)
Source code(zip)
v2.3.0(Oct 26, 2021)
Major Features and Improvements

Refactor posterior base class to support other model types. (#1695)

Add new posterior class to enable faster predictions from the GPR/SGPR models. (#1696, #1711)

Construct Parameters from other Parameters and retain properties. (#1699)

Add CGLB model (#1706)

Bug Fixes and Other Changes

Fix unit test failure when using TensorFlow 2.5.0 (#1684)

Upgrade black formatter to version 20.8b1 (#1694)

Remove erroneous DeprecationWarnings (#1693)

Fix SGPR derivation (#1688)

Fix tests which fail with TensorFlow 2.6.0 (#1714)

Thanks to our Contributors

This release contains contributions from:

johnamcleod, st--, Andrew878, tadejkrivec, awav, avullo
Source code(tar.gz)
Source code(zip)
v2.2.1(Apr 29, 2021)

Bug Fixes

Bugfix for creating the new posterior objects with PrecomputeCacheType.VARIABLE.
Source code(tar.gz)
Source code(zip)
v2.2.0(Apr 27, 2021)
Release 2.2.0

The main focus of this release is the new "Posterior" object introduced by PR #1636, which allows for a significant speed-up of post-training predictions with the SVGP model (partially resolving #1599).

For end-users, by default nothing changes; see Breaking Changes below if you have written your own implementations of gpflow.conditionals.conditional.

After training an SVGP model, you can call model.posterior() to obtain a Posterior object that precomputes all quantities not depending on the test inputs (e.g. Choleskty of Kuu), and provides a posterior.predict_f() method that reuses these cached quantities. model.predict_f() computes exactly the same quantities as before and does not give any speed-up.

gpflow.conditionals.conditional() forwards to the same "fused" code-path as before.

Breaking Changes

gpflow.conditionals.conditional.register is deprecated and should not be called outside of the GPflow core code. If you have written your own implementations of gpflow.conditionals.conditional(), you have two options to use your code with GPflow 2.2:

Temporary work-around: Instead of gpflow.models.SVGP, use the backwards-compatible gpflow.models.svgp.SVGP_deprecated.

Convert your conditional() implementation into a subclass of gpflow.posteriors.AbstractPosterior, and register get_posterior_class() instead (see the "Variational Fourier Features" notebook for an example).

Known Caveats

The Posterior object is currently only available for the SVGP model. We would like to extend this to the other models such as GPR, SGPR, or VGP, but this effort is beyond what we can currently provide. If you would be willing to contribute to those efforts, please get in touch!

The Posterior object does not currently provide the GPModel convenience functions such as predict_f_samples, predict_y, predict_log_density. Again, if you're willing to contribute, get in touch!

Thanks to our Contributors

This release contains contributions from:

@stefanosele, @johnamcleod, @st--
Source code(tar.gz)
Source code(zip)
v2.1.5(Apr 15, 2021)
Known Caveats

GPflow requires TensorFlow >= 2.2.

Deprecations

The gpflow.utilities.utilities submodule has been deprecated and will be removed in GPflow 2.3. User code should access functions directly through gpflow.utilities instead (#1650).

Major Features and Improvements

Improves compatibility between monitoring API and Scipy optimizer (#1642).

Adds _add_noise_cov method to GPR model class to make it more easily extensible (#1645).

Bug Fixes

Fixes a bug in ModelToTensorBoard (#1619) when max_size=-1 (#1619)

Fixes a dynamic shape issue in the quadrature code (#1626).

Fixes #1651, a bug in fully_correlated_conditional_repeat (#1652).

Fixes #1653, a bug in the "fallback" code path for multioutput Kuf (#1654).

Fixes a bug in the un-whitened code path for the fully correlated conditional function (#1662).

Fixes a bug in independent_interdomain_conditional (#1663).

Fixes an issue with the gpflow.config API documentation (#1664).

Test suite

Fixes the test suite for TensorFlow 2.4 / TFP 0.12 (#1625).

Fixes mypy call (#1637).

Fixes a bug in test_method_equivalence.py (#1649).

Thanks to our Contributors

This release contains contributions from:

johnamcleod, st--, vatsalaggarwal, sam-willis, vdutor
Source code(tar.gz)
Source code(zip)
v2.1.4(Dec 1, 2020)
Improvements

Replace len(inducing_variable) with inducing_variable.num inducing property (#1594). Adds support for inducing variables with dynamically changing shape (compatibility with tf.function). (Resolves #1578.)

HeteroskedasticTFPConditional should construct tensors at class-construction, not at module-import time (#1598).

Source code(tar.gz)
Source code(zip)
v2.1.3(Oct 7, 2020)
Improvements

kullback_leiblers.gauss_kl: when running TensorFlow >= 2.2, we avoid a tf.tile() that was a workaround for broadcasting issues in TensorFlow <= 2.1; the workaround is now only executed when running in TensorFlow 2.1 (#1585).

Bugfixes

model.predict_y/predict_log_density now explicitly raise NotImplementedError when passed full_(output_)cov=True (#1582). Actual implementation of the full_(output_)cov=True case remains an open issue (#1461).

bugfix for conditionals.multioutput.util.independent_interdomain_conditional (#1583).

Docs and infrastructure

CI/infrastructure updates (#1580, #1581, #1587, #1592).

heteroskedastic notebook cleaned up, links added (#1579).

intro_to_gpflow2 notebook updated to account for saved_model now working (#1586).

Source code(tar.gz)
Source code(zip)
v2.1.2(Sep 30, 2020)
New features and improvements

Likelihoods now use the new Quadrature class (#1566, #1571)

Multi-latent likelihoods for heteroskedastic models (#1559)

isort for consistent imports (#1572)

Bugfixes

Fix-up to our workaround for https://github.com/tensorflow/probability/issues/547 for tensorflow-probability==0.11.1 (#1574)

Add missing packaging dependency (#1568)

Deprecations

You should no longer use likelihood.num_gauss_hermite_points. Interact with its quadrature attribute instead.

Source code(tar.gz)
Source code(zip)
v2.1.1(Sep 14, 2020)
This patch release restores full compatibility with TensorFlow 2.3.

Known installation issue: if import gpflow gives you an ImportError, please install the "packaging" package manually (pip install packaging). Fixed by #1568, which will be included in the next patch release (GPflow 2.1.2).

Known bug: The deepcopy workaround for https://github.com/tensorflow/probability/issues/547 turned out not to be compatible with tensorflow-probability==0.11.1 which was released on 29 September 2020 on PyPI. We fixed this in #1574, which will be included in the following patch release (GPflow 2.1.2).

Improvements

Enable compatibility with TensorFlow 2.3 and tensorflow_probability 0.11.0 (#1553)

Recommend pip --use-feature=2020-resolver for installation

Ensure all gpflow code files contain copyright & license note (#1480)

Bugfixes

Fix SGPR upper bound for non zero mean functions (#1560)

Fix broadcasting for constant and zero mean functions (#1550)

Source code(tar.gz)
Source code(zip)
v2.1.0(Aug 27, 2020)
Release 2.1.0

Dependency constraints: To work around incompatibilities in our TensorFlow dependencies, GPflow 2.1.0 has the following new version requirements (#1522, #1537, #1551): tensorflow<2.3, tensorflow_probability>=0.10.1,<0.11, cloudpickle==1.3.0. We have restored compatibility with TensorFlow 2.3 and tensorflow_probability>=0.11 in the GPflow 2.1.1 patch release.

Key improvements

gpflow.Parameter finally plays well with TensorFlow's saving; instead of pretending to be a tf.Variable, we're now building on top of tfp.util.TransformedVariable (#1518) - see backwards incompatibilities below

Gauss-Hermite quadrature code was heavily refactored. The old function gpflow.quadrature.ndiagquad has been replaced by the gpflow.quadrature.NDiagGHQuadrature class - we strongly recommend upgrading (note that the dimensions of quadrature points have been moved from the end to the start for better broadcasting); ndiagquad will eventually be deprecated (#1505, #1542).

Minor changes

Add base_conditional_with_lm function that gets passed cholesky(K) instead of computing it explicitly (#1528)

Update to use stock rather than custom Docker image for CI (#1545)

Check dependency versions are valid on CI (#1536)

Docs: Readme updated to include new project that is using GPflow (#1530)

Bug Fixes

Restore pytest-xdist compatibility for parallelizing local tests (#1541)

Fix for quadrature failure mode when autograph was set to False (#1548)

Fix formatting in docs (intro.md) and restore link removed by #1498 (#1520)

Fix bug in varying_noise notebook (#1526)

Fix: separate_independent_conditional now correctly handles q_sqrt=None. (#1533)

Backwards incompatibilities

gpflow.Parameter is no longer pretending to be a tf.Variable, but instead subclasses tensorflow_probability's TransformedVariable. Several of the Variable-specific attributes/methods disappeared. Specifically, if you had previously called a Parameter's value() or read_value() methods, you should simply remove these.

Source code(tar.gz)
Source code(zip)
v2.0.5(Jun 11, 2020)
Improvements

gpflow.utilities.set_trainable() can now handle an Iterable of tf.Modules in one go (#1486)

Clarification: the NaturalGradient optimizer does not handle the diagonal (q_diag=True) case; emphasised this in the documentation and added shape checks (#1489)

Added type hints (#1487)

Potentially backwards-incompatible: gpflow.optimizers.Scipy: the callable passed in as step_callback is now passed the arguments (step, variables, values) by position, no longer by name (#1454)

Minor documentation improvements (#1488, #1492, #1498, #1503)

Bug fixes

multi-output combination kernels now support the presliced argument (#1495)

GPR can now be used with tf.Variables holding the data (#1490)

fixed indentation error in Makefile (#1482)

Source code(tar.gz)
Source code(zip)
v2.0.4(May 20, 2020)
Improvements

Add type hints to gpflow.base module (https://github.com/GPflow/GPflow/pull/1427)

Enable mypy checking on CI (https://github.com/GPflow/GPflow/pull/1471)

Bugfixes

TensorFlow Probability 0.10 introduces a new _parameters attribute in tfp.bijectors.Bijector classes. That attribute contains a self-reference that broke module traversing. In this fix, module traverse ignores _parameters attribute.

Handcrafted utilities.deepcopy was improved using memorization argument of python's copy.deepcopy method. The change prevents double copying of objects. (https://github.com/GPflow/GPflow/pull/1476)

Source code(tar.gz)
Source code(zip)
v2.0.3(May 13, 2020)

This release contains an update to setup.py for TensorFlow 2.2, fixing PyPi issue (#1460).
Source code(tar.gz)
Source code(zip)
v2.0.2(May 7, 2020)
Refactor of Natural Gradient optimizer implementation to be more easily extendable and to more efficiently handle updating several variational distributions simultaneously (#1443)

Change function signature of sample_mvn to take full_cov boolean instead of cov_structure string (#1448)

Fixes bug in Changepoint kernel evaluation (#1446)

Fixes #1439 (TensorFlow error when using larger datasets with tf.function compilation) in GPflow models that store data by converting it to tensors first, introduces data_input_to_tensor utility function (#1452, #1458)

Workaround for bug in TensorFlow <= 2.2.0rc3 in the leading_transpose utility function (#1442)

Improved representation of GPflow objects in IPython shell and Jupyter notebooks (#1453)

Minor cleanups and added type hints (#1421, #1438, #1441, #1450)

Source code(tar.gz)
Source code(zip)
v2.0.1(Apr 15, 2020)
Release notes:

Improve structure of likelihoods subdirectory (#1416)

Update README.md (#1401) and GPflow 2 upgrade guide (#1414)

Improved handling of invalid values for constrained Parameters (#1408)

Improvements on types/function annotations (#1406, #1420)

Documentation improvements (metalearning with GPs: #1382, coregionalization notebook: #1402, MCMC notebook: #1410, intro to gpflow with tensorflow 2: #1413)

Minor documentation fixes (#1429, #1430, #1433)

Fix: move matplotlib import inside ImageToTensorBoard (#1399)

Fix: tf.function compilation of ndiagquad (#1418)

Fix: cache tensorboard file writers and re-use them (#1424)

Source code(tar.gz)
Source code(zip)
v2.0.0(Mar 31, 2020)
GPflow 2.0 supports TF2.1 and does not support older versions TF1.x and TF2.0. GPflow stops accepting bug reports and PRs to the GPflow develop-1.0 branch. Please check the “GPflow 2 upgrade guide” and “Intro to GPflow with TensorFlow 2” notebooks in the documentation.

Parameter behaves as a tf.Tensor

Parameter has an option to set a prior on constrained or unconstrained space (#1177)

Parameter uses bijectors from TensorFlow Probability as a transformation. A positive bijector is provided in gpflow.utilities that can be configured to exp or softplus and has a variable lower bound.

Parameter doesn't have the setter for the trainable attribute. New utility function set_trainable added as a replacement.

Add GPflow config module for global settings: default float and integer types, default jitter, default positive transformation, minimal positive value, default printing format. Default settings can be changed using environment variables. The gpflowrc file is not supported anymore (#1148, #1267, #1135)

Add GPflow utility module. This module has many useful functions: custom operations like leading_transpose and broadcasting_elementwise, utility functions like customized deepcopy for tf.Module objects, freeze method that turns mutable models to immutable copies, to_default_float and many more (#1293)

Likelihoods have a more tightly defined interface, increasing shape robustness and making it easier to incorporate non-scalar likelihoods (#1334)

Add new Scipy optimizer that mimics standard scipy.minimize interface.

Remove custom GPflow optimizers and HMC implementation. A user can exploit standard TensorFlow optimizers and TensorFlow Probability MCMC sampling methods accordingly (#1100)

Add new monitor implementation (#1344)

Refactor stationary kernels (#1088)

Add Convolutional kernel (#996, #1034)

Add generalized Periodic kernel (#1158)

Add ChangePoint kernel (#1094)

Conditional calculations only take the lower triangle parts of q_sqrt into inconsideration (#1223)

Numerous improvements to notebooks and documentation:

Raw Jupyter notebook files replaced with jupytext python files (#1239)

Documentation generation in a separate repo (#1275)

Improved Bayesian model classes hierarchy reflecting the difference between data handling of models (#1276)

log_prior was renamed to log_prior_density (#1329)

predict_density was renamed to predict_log_density

full in the kernel.call was renamed to full_cov (#1319)

num_latent was renamed to num_latent_gps (#1310)

Thanks a lot to everyone who made GPflow 2.0 happen:

Artem Artemev, @awav S.T. John, @st-- Vincent Dutordoir, @vdutor Mark van der Wilk, @markvdw James Hensman, @jameshensman Eric Hambro, @condnsdmatters Joel Berkeley, @joelberkeley-pio Alexis Boukouvalas, @alexisboukouvalas Sergio Pascual Diaz, @spascual Vincent Adam, @thevincentadam Fergus Simpson, @frgsimpson @ilia-kats @marcoadurno Joe Hall, @jahall Tadej Krivec, @tadejkrivec (SGPR theory derivation fix) Alexandra Hayes, @akhayes; James Leedham, (copyediting on notebooks) Nicolas Durrande, @NicolasDurrande; Felix Leibfried, @fleibfried; Jordi Graumo, @jordigraumo (porting notebooks to gpflow 2.0) Neil Ferguson, @pio-neil (GPflow 2 upgrade guide)
Source code(tar.gz)
Source code(zip)
2.0.0-rc1(Nov 14, 2019)
GPflow works with TensorFlow 2.0!

Features:

No static graphs, pythonic way of writing the code

Refactored GPflow configuration

Updated model printing

Fast multiple assign

Convolutional kernel

ChangePoint kernel

Source code(tar.gz)
Source code(zip)
v1.5.1(Nov 3, 2019)
Features and improvements:

Copyedits in documentation and notebooks

Make likelihood quadrature tests comprehensive (#1081)

Bugfixes:

Fix an issue with graph pollution in the monitor task ImageToTensorboardTask (#1041)

Source code(tar.gz)
Source code(zip)
v1.5.0(Aug 13, 2019)
Features and improvements:

Reusable ScipyOptimizer (#1011)

Documentation copyedits (#1019, #1013-1017)

Convolutional kernel (#996)

Saver works with file-like objects as well as path names (#912)

Bugfixes:

The misc.initialize_variables() does not keep adding ops to the graph (#1001)

Variables restored in monitoring notebook (#1003)

Support TensorFlow 1.14.0 (#997)

Shape checks for multivariate_normal (#964)

Source code(tar.gz)
Source code(zip)
v1.4.1(Jun 17, 2019)
Features, bugfixes and improvements:

Minor performance improvement: pass cholesky to the gauss_kl (#933)

Documentation revamp:

Organize documentation and notebooks (#924)

GPflow glossary document (#923)

Fix AutoBuild metaclass for method signatures (#907)

CircleCI instead of TravisCI (#892)

Broadcasting conditionals via tf.broadcast_to (#895)

Support TensorFlow >= 1.12 (#877)

Source code(tar.gz)
Source code(zip)
v1.3.0(Oct 19, 2018)
Main features and improvements

Cleaning up stationary kernel implementations: now defined in terms of K_r or K_r2. (#827)

Support broadcasting over arbitrarily many leading dimensions for kernels and conditional. (#829)

Analytic expectation of the cross-covariance between different RBF kernels. (#754)

New MixedKernelSeparateMof feature class for multi-output GPs. (#830)

The sample_conditional returns mean and var as well as samples, and can generate more than one sample. (#836)

Support monitoring with ScipyOptimizer. (#856)

Bug fixes

Fix bug in ndiag_mc for multi-dimensional kwargs. (#813)

Fix parameter.trainable to be a property. (#814)

Remove references to six module. (#816)

Fix tf.control_dependencies in likelihoods. (#821)

Fix active_dims for slice type. (#840)

Source code(tar.gz)
Source code(zip)