Gaussian processes in TensorFlow

Overview

CircleCI Coverage Status Documentation Status Slack Status

Website | Documentation (release) | Documentation (develop) | Glossary

Table of Contents

What does GPflow do?

GPflow is a package for building Gaussian process models in Python. It implements modern Gaussian process inference for composable kernels and likelihoods.

GPflow 2.1 builds on TensorFlow 2.2+ and TensorFlow Probability for running computations, which allows fast execution on GPUs.

The online documentation (develop)/(master) contains more details.

Maintainers

It was originally created by James Hensman and Alexander G. de G. Matthews. It is now actively maintained by (in alphabetical order) Artem Artemev, Mark van der Wilk, ST John, and Vincent Dutordoir. GPflow would not be the same without the community. We are grateful to all contributors who have helped shape GPflow.

GPflow is an open source project. If you have relevant skills and are interested in contributing then please do contact us (see "The GPflow community" section below).

Installation

Requirements

GPflow depends on both TensorFlow (TF, version ≥ 2.2) and TensorFlow Probability (TFP, version ≥ 0.10.1). We support Python ≥ 3.6.

NOTE: TensorFlow Probability releases are tightly coupled to TensorFlow, e.g. TFP 0.12 requires TF>=2.4, TFP 0.11 requires TF>=2.3, and TFP 0.10 requires TF>=2.2. Unfortunately, this is not specified in TFP's dependencies. So if you already have an (older) version of TensorFlow installed, GPflow will pull in the latest TFP, which will be incompatible. If you get errors such as ImportError: This version of TensorFlow Probability requires TensorFlow version >= 2.4, you have to either upgrade TensorFlow (pip install -U tensorflow) or manually install an older version of the tensorflow_probability package.

Latest (stable) release from PyPI

pip install gpflow

Latest (bleeding-edge) source from GitHub

Be aware that the develop branch may change regularly, and new commits may break your code.

In a check-out of the develop branch of the GPflow GitHub repository, run

pip install -e .

Alternatively, you can install the latest GitHub develop version using pip:

pip install git+https://github.com/GPflow/GPflow.git@develop#egg=gpflow

This will automatically install all required dependencies.

Getting Started with GPflow 2.0

There is an "Intro to GPflow 2.0" Jupyter notebook; check it out for details. To convert your code from GPflow 1 check the GPflow 2 upgrade guide.

The GPflow Community

Getting help

Bugs, feature requests, pain points, annoying design quirks, etc: Please use GitHub issues to flag up bugs/issues/pain points, suggest new features, and discuss anything else related to the use of GPflow that in some sense involves changing the GPflow code itself. You can make use of the labels such as bug, discussion, feature, feedback, etc. We positively welcome comments or concerns about usability, and suggestions for changes at any level of design.

We aim to respond to issues promptly, but if you believe we may have forgotten about an issue, please feel free to add another comment to remind us.

"How-to-use" questions: Please use Stack Overflow (gpflow tag) to ask questions that relate to "how to use GPflow", i.e. questions of understanding rather than issues that require changing GPflow code. (If you are unsure where to ask, you are always welcome to open a GitHub issue; we may then ask you to move your question to Stack Overflow.)

Slack workspace

We have a public GPflow slack workspace. Please use this invite link if you'd like to join, whether to ask short informal questions or to be involved in the discussion and future development of GPflow.

Contributing

All constructive input is gratefully received. For more information, see the notes for contributors.

Projects using GPflow

Projects building on GPflow and demonstrating its usage are listed below. The following projects are based on the current GPflow 2.x release:

Project Description
Trieste Bayesian optimization with TensorFlow, with out-of-the-box support for GPflow (2.x) models.
VFF Variational Fourier Features for Gaussian Processes (GPflow 2.x version)
BranchedGP Gaussian processes with branching kernels.
VBPP Implementation of "Variational Bayes for Point Processes".
Gaussian Process Regression on Molecules GPs to predict molecular properties by creating a custom-defined Tanimoto kernel to operate on Morgan fingerprints

If you would like your project listed here, let us know - or simply open a pull request that adds your project to the table above!

The following projects build on older versions of GPflow (pre-2020); we encourage their authors to upgrade to GPflow 2.

Project Description
GPflowOpt Bayesian Optimization using GPflow (stable release requires GPflow 0.5).
Doubly-Stochastic-DGP Deep Gaussian Processes with Doubly Stochastic Variational Inference.
widedeepnetworks Measuring the relationship between random wide deep neural networks and GPs.
orth_decoupled_var_gps Variationally sparse GPs with orthogonally decoupled bases
kernel_learning Implementation of "Differentiable Compositional Kernel Learning for Gaussian Processes".
DGPs_with_IWVI Deep Gaussian Processes with Importance-Weighted Variational Inference
kerndisc Library for automated kernel structure discovery in univariate data
Signature covariances kernels for (time)series as inputs
Structured-DGP Adding more structure to the variational posterior of the Doubly Stochastic Deep Gaussian Process

Version Compatibility

GPflow heavily depends on TensorFlow and as far as TensorFlow supports forward compatibility, GPflow should as well. The version of GPflow can give you a hint about backward compatibility. If the major version has changed then you need to check the release notes to find out how the API has been changed.

Unfortunately, there is no such thing as backward compatibility for GPflow models, which means that a model implementation can change without changing interfaces. In other words, the TensorFlow graph can be different for the same models from different versions of GPflow.

TensorFlow 1.x and GPflow 1.x

We have stopped development and support for GPflow based on TensorFlow 1. The latest release supporting TensorFlow 1 is v1.5.1. Documentation and tutorials will remain available.

Citing GPflow

To cite GPflow, please reference the JMLR paper. Sample Bibtex is given below:

@ARTICLE{GPflow2017,
  author = {Matthews, Alexander G. de G. and {van der Wilk}, Mark and Nickson, Tom and
	Fujii, Keisuke. and {Boukouvalas}, Alexis and {Le{\'o}n-Villagr{\'a}}, Pablo and
	Ghahramani, Zoubin and Hensman, James},
    title = "{{GP}flow: A {G}aussian process library using {T}ensor{F}low}",
  journal = {Journal of Machine Learning Research},
  year    = {2017},
  month = {apr},
  volume  = {18},
  number  = {40},
  pages   = {1-6},
  url     = {http://jmlr.org/papers/v18/16-537.html}
}

Since the publication of the GPflow paper, the software has been significantly extended with the framework for interdomain approximations and multioutput priors. We review the framework and describe the design in an arXiv paper, which can be cited by users.

@article{GPflow2020multioutput,
  author = {{van der Wilk}, Mark and Dutordoir, Vincent and John, ST and
            Artemev, Artem and Adam, Vincent and Hensman, James},
  title = {A Framework for Interdomain and Multioutput {G}aussian Processes},
  year = {2020},
  journal = {arXiv:2003.01115},
  url = {https://arxiv.org/abs/2003.01115}
}
Comments
  • Matrix Triangular Solve does not require CUSolver for GPU

    Matrix Triangular Solve does not require CUSolver for GPU

    I was just thinking about this on a walk. See section 3.4.6 of the linked document

    https://developer.nvidia.com/sites/default/files/akamai/cuda/files/Misc/mygpu.pdf

    Therefore our current bottleneck for SVGP on float32 can be moved to GPU without the difficult changes to the stream executor discussed here:

    https://github.com/tensorflow/tensorflow/issues/2217

    This is important because removing the MatrixTriangularSolve bottleneck would speed up GPU performance by about a factor of 3.

    I guess CUSOLVER is only required for the Cholesky decomposition, which would be one of our next bottlenecks.

    A look at Tom Nixon's fork confirms the GPU matrix triangular solve implementation only calls CUBLAS not CUSOLVER.

    https://github.com/c0g/tensorflow/blob/master/tensorflow/core/kernels/matrix_triangular_solve_op.cc

    I wish I had time to work on this right now but I don't unfortunately. Thought everyone should know straight away, anway.

    discussion 
    opened by alexggmatthews 60
  • Kernel expectations

    Kernel expectations

    Ok, so again not ready to be merged, but a start to what we're discussing in #91. There are lots of things up for discussion and TODOs to merge the existing psi calculations in here. I'll try to summarise.

    Notes on the commit here:

    • I have implemented exKxz which is the expectation <x_t K_{x_{t-1}, Z}>_q_{x_{t-1:t}}. This a necessary piece for GPSSMs, which I'm interested in. ~~It should collapse to eKxz for delta q(x_t).~~ (don't think this is true anymore).
    • The function exKxz computes the above expectation for an entire time series. So give it N+1 means (Xmu N+1 x D) and N+1 covariances and cross-covariances (Xcov 2xN+1xDxD). However, in order for this to work the cross covariance in Xcov[1, n-1, :, :] needs to be consistent with the covariance in Xcov[0, n, :, :]. This is ensured by theTriDiagonalBlockReptransform that I have added inetransforms.py`.
    • ~~I haven't implemented eKxz yet.~~ This is now done.
    • ~~I have Monte Carlo tests for all the expectations. However, the one for exKxz has high variance. I would appreciate help here.~~

    I added TriDiagonalBlockRep along with the tests mostly because I wanted to upload something quickly. I feel like it may be a little outside of scope for GPflow, and it's not quite finished. Essentially it's a way to freely parameterise a covariance matrix with a Markovian structure (i.e. the precision is block tri diagonal). If this is something that could be useful / isn't minded in GPflow, I'll leave it in. Otherwise I'll rework test_ekern.py to not need it.

    Overall things to do and decide:

    • ~~Should we keep kernels with expectations separate from the original kernels? I.e. should the functionality in ekernels.py' be merged withkernels.py`?~~ No, keep them separate.
    • ~~Add quadrature for 1D kernels in the kernel base class.~~
    • ~~Add expectations to Linear kernel.~~
    • ~~Sort out sum and product kernels.~~
    • ~~Add a check that allows for diagonal q(X) as well.~~
    • ~~Make tests more reliable. Perhaps we can do some low dimensional quadrature instead of relying on monte carlo? Currently variance causes things to fail sometimes.~~
    opened by markvdw 46
  • Add MLP kernel

    Add MLP kernel

    This pull request adds an implementation of the MLP kernel as used in GPy, together with a few tests. If there is interest to merge this PR, I can add more information to the docstring or maybe more tests. This code is also not tested very thoroughly yet.

    opened by mrksr 29
  • Example of end-to-end training with GPFlow plus a neural net coded in tensorflow

    Example of end-to-end training with GPFlow plus a neural net coded in tensorflow

    Hello,

    Is there any example on how to implement a neural network using tensorflow plus a final 'layer' implementing a GP process. Something similar to the architecture presented in: "Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks" here

    I would like to be able to train the resulting model end-to-end. I suppose using the variational model it could be possible to optimize the GP parameters plus the NN weights using gradient descent, all at the same time, but I do not know how to code to connect the two models (tensorflow NN and GPflow GP)

    Thank you

    opened by mlopezm 28
  • GPflow has become considerably slower on GPU with some recent changes.

    GPflow has become considerably slower on GPU with some recent changes.

    I have raised this as a bug because as you will see the effect is large.

    The main changes I can see are:

    1. The addition of user_ops for VecToTri and TriToVec
    2. The use of batch matrix triangular solve.

    The git hashes I'm comparing are:

    new with changes 23b47f590806f0da3cb71916d68674f8bee4d875 old without changes 9c2faf62d04f8ec2ab35396553b5115d92d3c84b

    I'm running MNIST training with a minibatch size of 1000.

    GPU operation is 70% slower between these two hashes. CPU operation is about the same. Did we observe a speed up when we merged these?

    The new profiling code #174 allows us to give a detailed breakdown of the slow down. user_ops_comparison.tar.gz

    The can be viewed in chrome using the GUI found at chrome://tracing .

    We find that the batch matrix triangular solve does not parallelize as well as the triangular_solve which is arguably a flaw in TensorFlow itself.

    We find that the VecToTri and TriToVec ops are currently very slow relatively considering they just reshape something.

    What should we do about this?

    I would argue we should find out exactly which recent changes are causing this and revert them as soon as possible. The new code looks really great and requires considerably skill but this is showing us that it isn't ready for master yet IMHO. The new user ops also make installation more difficult. We want our users at any moment to have the best available experience and we have to be dedicated to this if we are going to achieve our ambitions for the software.

    Longer term I argue that we should move the code to pull requests and expedite #174 #171 and #170

    @markvdw @jameshensman thoughts?

    bug 
    opened by alexggmatthews 28
  • serializing GPflow objects

    serializing GPflow objects

    This branch is perhaps mis-named, it does two things, both addressing #48

    First, the ability to get all the parameters in a models in a dictionary.

    Second, the ability to pickle GPflow objects.

    This is very much up for discussion, and we may want to separate the two features. I think they're both useful though, and both address the same open question.

    See also #56

    import numpy as np
    import GPflow
    import pickle
    X, Y = np.random.randn(2, 10, 1)
    m = GPflow.sgpr.SGPR(X, Y, GPflow.kernels.Matern32(1), Z=X)
    s1 = pickle.dumps(m)
    m._compile()
    s2 = pickle.dumps(m)
    m1 = pickle.loads(s1)
    m2 = pickle.loads(s2)
    print m.compute_log_likelihood()
    print m1.compute_log_likelihood()
    print m2.compute_log_likelihood()
    
    help wanted 
    opened by jameshensman 28
  • vector Gaussian likelihood variance for varying measurement uncertainty.

    vector Gaussian likelihood variance for varying measurement uncertainty.

    I would be very useful to have a vector variance in the Gaussian likelihood, as it is very often the case that measurements do not have a constant uncertainty.

    opened by Joshuaalbert 26
  • Fix covariance overparameterisation

    Fix covariance overparameterisation

    NB: This is nowhere near ready to commit yet. I just want some comments (from @alexggmatthews in particular).

    This is a start to work to fix https://github.com/GPflow/GPflow/issues/49. This needs a new TensorFlow op to turn a vector into a triangular matrix. On the plus side, it's possible to do this without having to maintain a special fork of TensorFlow. With the TensorFlow binaries installed, custom ops can be compiled in place and then imported as a standard Python module.

    The op is in /GPflow/tfops/vec_to_tri.cc, and needs to be compiled with the command: Linux: g++ -std=c++11 -shared zero_out.cc -o zero_out.so -fPIC -I $(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())') MacOS: g++ -undefined dynamic_lookup -std=c++11 -shared zero_out.cc -o zero_out.so -fPIC -I $(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')

    There's a small example script calling the custom op in /op.py.

    I've basically just adapted the TensorFlow custom op example, and need some pointers on how to proceed. @alexggmatthews, what resources did you use to find out the style used in TensorFlow and how to manipulate the Tensor objects etc? Right now I'm modifying individual elements in a very roundabout way, by first flattening them.

    Next steps are to properly template the ops to ensure compatibility with more data types, adding gradients, tests etc...

    enhancement please review and merge 
    opened by markvdw 24
  • Issues in GP regression with the Matern Kernel. Is RBF more numerically stable than the Matern Kernel?

    Issues in GP regression with the Matern Kernel. Is RBF more numerically stable than the Matern Kernel?

    I am doing GP regression with some real data dim(X)=[N,3] and dim[Y] = [N,1]. When I use the RBF Kernel it works fine when I use the Matern kernel it gives me some errors, probably in computing the cholesky during the optimazation. Here an idea of my code:

    import GPflow
    
    m = GPflow.gpr.GPR(X, Y, kern=GPflow.kernels.RBF(3))
    m.optimize()
    
    
    m2 = GPflow.gpr.GPR(X, Y, kern=GPflow.kernels.Matern52(3)+ GPflow.kernels.White(input_dim=3))
    m2.optimize()
    

    This is the error that I obtain

    2017-08-21 17:49:05.142629: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
    2017-08-21 17:49:05.142639: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
    2017-08-21 17:49:05.142641: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
    2017-08-21 17:49:05.142643: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
    2017-08-21 17:49:05.142645: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
    2017-08-21 17:49:05.246520: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:893] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
    2017-08-21 17:49:05.246789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:940] Found device 0 with properties: 
    name: GeForce GTX 1080
    major: 6 minor: 1 memoryClockRate (GHz) 1.86
    pciBusID 0000:01:00.0
    Total memory: 7.92GiB
    Free memory: 7.16GiB
    2017-08-21 17:49:05.246798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:961] DMA: 0 
    2017-08-21 17:49:05.246801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] 0:   Y 
    2017-08-21 17:49:05.246806: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
    2017-08-21 17:49:06.786071: I tensorflow/core/kernels/cuda_solvers.cc:137] Creating CudaSolver handles for stream 0x510a320
    2017-08-21 17:49:11.326938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
    2017-08-21 17:49:13.169500: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Got info = 49 for batch index 0, expected info = 0. Debug_info =potrf
    Traceback (most recent call last):
      File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1596, in <module>
        globals = debugger.run(setup['file'], None, None, is_module)
      File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1023, in run
        pydev_imports.execfile(file, globals, locals)  # execute the script
      File "/homes/romeres/Projects/1DModelCodes/syncinputs_v2.py", line 177, in <module>
        m2.optimize()
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 223, in optimize
        return self._optimize_np(method, tol, callback, maxiter, **kw)
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 308, in _optimize_np
        options=options)
      File "/usr/lib/python2.7/dist-packages/scipy/optimize/_minimize.py", line 447, in minimize
        callback=callback, **options)
      File "/usr/lib/python2.7/dist-packages/scipy/optimize/lbfgsb.py", line 330, in _minimize_lbfgsb
        f, g = func_and_grad(x)
      File "/usr/lib/python2.7/dist-packages/scipy/optimize/lbfgsb.py", line 278, in func_and_grad
        f = fun(x, *args)
      File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 289, in function_wrapper
        return function(*(wrapper_args + args))
      File "/usr/lib/python2.7/dist-packages/scipy/optimize/optimize.py", line 63, in __call__
        fg = self.fun(x, *args)
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 43, in __call__
        f, g = self._objective(x)
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 161, in obj
        feed_dict=feed_dict)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
        run_metadata_ptr)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
        feed_dict_string, options, run_metadata)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
        target_list, options, run_metadata)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
        raise type(e)(node_def, op, message)
    tensorflow.python.framework.errors_impl.InvalidArgumentError: Got info = 49 for batch index 0, expected info = 0. Debug_info =potrf
    	 [[Node: name.build_likelihood/Cholesky = Cholesky[T=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/gpu:0"](name.build_likelihood/add)]]
    	 [[Node: name.build_likelihood/Cholesky/_27 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_394_name.build_likelihood/Cholesky", tensor_type=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
    
    Caused by op u'name.build_likelihood/Cholesky', defined at:
      File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1596, in <module>
        globals = debugger.run(setup['file'], None, None, is_module)
      File "/homes/romeres/.local/share/umake/ide/pycharm/helpers/pydev/pydevd.py", line 1023, in run
        pydev_imports.execfile(file, globals, locals)  # execute the script
      File "/homes/romeres/Projects/1DModelCodes/syncinputs_v2.py", line 177, in <module>
        m2.optimize()
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 223, in optimize
        return self._optimize_np(method, tol, callback, maxiter, **kw)
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 284, in _optimize_np
        self._compile()
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/model.py", line 133, in _compile
        f = self.build_likelihood() + self.build_prior()
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/scoping.py", line 43, in runnable
        return f(*args, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/GPflow-0.3.8-py2.7.egg/GPflow/gpr.py", line 60, in build_likelihood
        L = tf.cholesky(K)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_linalg_ops.py", line 227, in cholesky
        result = _op_def_lib.apply_op("Cholesky", input=input, name=name)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
        op_def=op_def)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
        original_op=self._default_original_op, op_def=op_def)
      File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1269, in __init__
        self._traceback = _extract_stack()
    
    InvalidArgumentError (see above for traceback): Got info = 49 for batch index 0, expected info = 0. Debug_info =potrf
    	 [[Node: name.build_likelihood/Cholesky = Cholesky[T=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/gpu:0"](name.build_likelihood/add)]]
    	 [[Node: name.build_likelihood/Cholesky/_27 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_394_name.build_likelihood/Cholesky", tensor_type=DT_DOUBLE, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
    
    
    Process finished with exit code 1
    

    I obtained this error from the PyCharm console but from terminal it is analogous. Line 177 corresponds to m2.optimize().

    As you can see I tried to add White noise to overcome possible numerical error. I also tried to constrain the hyperparameters within a certain range but the result did not change.

    m2.kern.matern52.lengthscales.transform = GPflow.transforms.Logistic(0.1, 10.)
    m2.kern.matern52.variance.transform = GPflow.transforms.Logistic(0.1, 10.)
    

    Am I missing something or is there a problem in the Matern kernel implementation? Thank you in advance.

    opened by DiegoRomeres 23
  • GPflow 2.0

    GPflow 2.0

    Hello Everyone!

    TensorFlow 2.0 is coming and will bring important changes. In short, TensorFlow will have function style execution by making Eager mode default, details are here. This is very exciting news, as with "dynamic-like" graph creation users will have more flexibility and be able to do more (e.g. proper for loops and conditions). I started awav/gpflow-2.0 branch as an experimental solution and eager mode playground, but eventually awav/gpflow-2.0 will be merged into the GPflow master. GPflow 2.0 branch is expected to be unstable for now.

    Any feedback and suggestions are welcome!

    Plans of what we would like to do:

    • At first, integrate GPflow with tensorflow_probabilities.
    • We are considering to support both current TensorFlow static graph execution and Eager modes. It is reasonable, but implies difficulties in implementation.
    • Unify saver interface with possibility to use TensorFlow (Keras) saver.
    • Support Keras monitoring as well.

    What awav/gpflow-2.0 provides now:

    • Project layout is cleaned and "optimized".
    • Autoflow methods are removed, there is no build_* functions.
    • @params_as_tensors are removed. There is no more such things as tf.Tensor which you can run via session.run and Parameter class is actually inherited from Eager Variable.
    • TensorFlow Bijectors are used as Parameter transforms.
    • TensorFlow distributions are used as Parameter priors.
    • Parameterized is renamed to Module

    There is a prototype example. For now only GPR and SVGP models are ported. In that file you will find rough example how to use SVGP and what you can do with GPflow 2.0.

    Thank you!

    discussion important announcement 
    opened by awav 22
  • Tracer and wrapped session for GPflow

    Tracer and wrapped session for GPflow

    Not to be merged yet.

    This PR currently allows one to switch on a profiling mode that stores a tracer file for calls to tf.session.run . It can be set to store a tracer file for each such call or only the last one.

    To do this there are two key concepts:

    1. TensorFlow itself has the ability to do this either for both CPU and GPU processes. They key commands are
    run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
    run_metadata = tf.RunMetadata()
    sess.run(res, options=run_options, run_metadata=run_metadata)
    tl = timeline.Timeline(run_metadata.step_stats)
    ctf = tl.generate_chrome_trace_format()
    with open('timeline.json', 'w') as f:
       f.write(ctf)
    
    
    1. TensorFlow (oddly) does not have the ability to do this every time with some constant run_options. Therefore I have wrapped all calls to session in GPflow. We can then use either a standard tf.Session or a special new class that has the same interface as the normal session but is running and storing the traces each time. I have done it this way to minimize the impact on the rest of the code.

    I can imagine we might want other interfaces for the profiler which is why I'm doing the PR a bit early.

    I still need to write unit tests and update the existing ones.

    An example script is provided in notebooks/example_profiler.py

    enhancement help wanted 
    opened by alexggmatthews 21
  • Noted cpython 3.8 bug

    Noted cpython 3.8 bug

    PR type: doc improvement

    Related issue(s)/PRs

    Summary

    The bug in cpython3.8 linked above is causing our benchmarking to fail. This isn't really fixable by us, so I've added a warning comment to prevent people banging their heads against this issue.

    Fully backwards compatible: yes

    PR checklist

    • [ ] New features: code is well-documented
      • [ ] detailed docstrings (API documentation)
      • [ ] notebook examples (usage demonstration)
    • [ ] The bug case / new feature is covered by unit tests
    • [ ] Code has type annotations
    • [x] Build checks
      • [x] I ran the black+isort formatter (make format)
      • [x] I locally tested that the tests pass (make check-all)
    • [ ] Release management
      • [ ] RELEASE.md updated with entry for this change
      • [ ] New contributors: I've added myself to CONTRIBUTORS.md
    opened by sc336 1
  • Check shape warning when importing gpflow

    Check shape warning when importing gpflow

    Running import gpflow in the latest release outputs the following warnings:

    In [1]: import gpflow
    /home/uri.granta/.pyenv/versions/3.7.13/envs/trieste/lib/python3.7/site-packages/gpflow/experimental/utils.py:43: UserWarning: You're calling gpflow.experimental.check_shapes.decorator.check_shapes which is considered *experimental*. Expect: breaking changes, poor documentation, and bugs.
      f"You're calling {name} which is considered *experimental*."
    /home/uri.granta/.pyenv/versions/3.7.13/envs/trieste/lib/python3.7/site-packages/gpflow/experimental/utils.py:43: UserWarning: You're calling gpflow.experimental.check_shapes.inheritance.inherit_check_shapes which is considered *experimental*. Expect: breaking changes, poor documentation, and bugs.
      f"You're calling {name} which is considered *experimental*."
    

    While it may be reasonable to output a warning when a user uses check shape for the first time, we probably shouldn't output it when gpflow itself does.

    bug 
    opened by uri-granta 1
  • Demonstrate best practices in the notebooks: `@tf.function`

    Demonstrate best practices in the notebooks: `@tf.function`

    Documentation/tutorial notebooks

    Are there any mistakes in the docs? Some notebooks (e.g. https://gpflow.github.io/GPflow/develop/notebooks/advanced/natural_gradients.html, there may be more, I did not check extensively) do not use @tf.function when using a non-Scipy optimizer (our Scipy class automatically wraps the optimisation in tf.function by default). This is significantly slower in practice, and for any real data sets you would want to ensure the tf.function-precompilation. The notebooks should demonstrate this best practice, so users do not wonder why GPflow is so slow...

    opened by st-- 0
  • Added _sparse_orthogonal_conditional

    Added _sparse_orthogonal_conditional

    Draft PR for taking the GPflow components from this GPflux PR into GPflow. The work is based on this paper.

    ============================

    PR type: bugfix / enhancement / new feature / doc improvement

    Related issue(s)/PRs:

    Summary

    Proposed changes

    • ...
    • ...
    • ...

    What alternatives have you considered?

    Minimal working example

    # Put your example code in here
    

    Release notes

    Fully backwards compatible: yes / no

    If not, why is it worth breaking backwards compatibility:

    PR checklist

    • [ ] New features: code is well-documented
      • [ ] detailed docstrings (API documentation)
      • [ ] notebook examples (usage demonstration)
    • [ ] The bug case / new feature is covered by unit tests
    • [ ] Code has type annotations
    • [ ] Build checks
      • [ ] I ran the black+isort formatter (make format)
      • [ ] I locally tested that the tests pass (make check-all)
    • [ ] Release management
      • [ ] RELEASE.md updated with entry for this change
      • [ ] New contributors: I've added myself to CONTRIBUTORS.md
    opened by sc336 1
  • turn jitter into an argument of inv_probit instead of hard-coded inside

    turn jitter into an argument of inv_probit instead of hard-coded inside

    Summary

    Turn jitter of inv_probit utility function into an argument. Add docstring.

    Release notes

    Fully backwards compatible: yes

    PR checklist

    • [x] New features: code is well-documented
      • [x] detailed docstrings (API documentation)
      • [ ] notebook examples (usage demonstration)
    • [ ] The bug case / new feature is covered by unit tests
    • [x] Code has type annotations
    • [ ] Build checks
      • [ ] I ran the black+isort formatter (make format)
      • [ ] I locally tested that the tests pass (make check-all)
    • [ ] Release management
      • [ ] RELEASE.md updated with entry for this change
      • [ ] New contributors: I've added myself to CONTRIBUTORS.md
    opened by st-- 0
Releases(v2.6.4)
  • v2.6.4(Dec 2, 2022)

    This is yet another bug-fix release.

    Bug Fixes and Other Changes

    • Fix to to_default_float to avoid losing precision when called with python floats.

    Thanks to our Contributors

    This release contains contributions from:

    ChrisMorter

    Source code(tar.gz)
    Source code(zip)
  • v2.6.3(Oct 13, 2022)

    Release 2.6.3 (next upcoming release in progress)

    This is yet another bug-fix release.

    Bug Fixes and Other Changes

    • Fix to check_shapes handling of tfp..._TensorCoercible.

    Thanks to our Contributors

    This release contains contributions from:

    jesnie

    Source code(tar.gz)
    Source code(zip)
  • v2.6.2(Oct 10, 2022)

    Release 2.6.2

    This is a bug-fix release, for compatibility with GPflux.

    Bug Fixes and Other Changes

    • Extract shapes of tfp.python.layers.internal.distribution_tensor_coercible._TensorCoercible.
    • Allow FallbackSeparateIndependentInducingVariables to have children with different shapes.
    • Allow input and output batches on GaussianQuadrature to be different.

    Thanks to our Contributors

    This release contains contributions from:

    jesnie

    Source code(tar.gz)
    Source code(zip)
  • v2.6.1(Sep 29, 2022)

    Release 2.6.1

    This is a bug-fixes release, due to problems with model saving in 2.6.0.

    Breaking Changes

    • Removed gpflow.utilities.ops.cast. Use tf.cast instead.

    Bug Fixes and Other Changes

    • Fixed bug related to tf.saved_model and methods wrapped in @check_shapes.
    • Some documentation formatting fixes.

    Thanks to our Contributors

    This release contains contributions from:

    jesnie

    Source code(tar.gz)
    Source code(zip)
  • v2.6.0(Sep 20, 2022)

    Release 2.6.0

    The major theme for this release is heteroskedastic likelihoods. Changes have unfortunately caused some breaking changes, but makes it much easier to use heteroskedastic likelihoods, either by plugging together built-in GPflow classes, or when writing your own. See our updated notebook, for examples on how to use this.

    Breaking Changes

    • All likelihood methods now take an extra X argument. If you have written custom likelihoods or you have custom code calling likelihoods directly you will need to add this extra argument.
    • On the CGLB model the xnew parameters has changed name to Xnew, to be consistent with the other models.
    • On the GPLVM model the variance returned by predict_f with full_cov=True has changed shape from [batch..., N, N, P] to [batch..., P, N, N] to be consistent with the other models.
    • gpflow.likelihoods.Gaussian.DEFAULT_VARIANCE_LOWER_BOUND has been replaced with gpflow.likelihoods.scalar_continuous.DEFAULT_LOWER_BOUND.
    • Change to InducingVariables API. InducingVariables must now have a shape property.
    • gpflow.experimental.check_shapes.get_shape.register has been replaced with gpflow.experimental.check_shapes.register_get_shape.
    • check_shapes will no longer automatically wrap shape checking in tf.compat.v1.flags.tf_decorator.make_decorator. This is likely to affect you if you use check_shapes with custom Keras models. If you require the decorator you can manually enable it with check_shapes(..., tf_decorator=True).

    Known Caveats

    • Shape checking is now, by default, disabled within tf.function. Use set_enable_check_shapes to change this behaviour. See the API documentation for more details.

    Major Features and Improvements

    • Improved handling of variable noise
      • All likelihood methods now take an X argument, allowing you to easily implement heteroskedastic likelihoods.
      • The Gaussian likelihood can now be parametrized by either a variance or a scale
      • Some existing likelihoods can now take a function (of X) instead of a parameter, allowing them to become heteroskedastic. The parameters are:
        • Gaussian variance
        • Gaussian scale
        • StudentT scale
        • Gamma shape
        • Beta scale
      • The GPR and SGPR can now be configured with a custom Gaussian likelihood, allowing you to make them heteroskedastic.
      • See the updated notebook.
      • gpflow.mean_functions has been renamed gpflow.functions, but with an alias, to avoid breaking changes.
    • gpflow.experimental.check_shapes
      • Can now be in three different states - ENABLED, EAGER_MODE_ONLY, and DISABLE. The default is EAGER_MODE_ONLY, which only performs shape checks when the code is not compiled. Compiling the shape checking code is a major bottleneck and this provides a significant speed-up for performance sensitive parts of the code.
      • Now supports multiple variable-rank dimensions at the same time, e.g. cov: [n..., n...].
      • Now supports single broadcast dimensions to have size 0 or 1, instead of only 1.
      • Now supports variable-rank dimensions to be broadcast, even if they're not leading.
      • Now supports is None and is not None as checks for conditional shapes.
      • Now uses custom function register_get_shape instead of get_shape.register, for better compatibility with TensorFlow.
      • Now supports checking the shapes of InducingVariables.
      • Now adds documentation to function arguments that has declared shapes, but no other documentation.
      • All of GPflow is now consistently shape-checked.
    • All built-in kernels now consistently support broadcasting.

    Bug Fixes and Other Changes

    • Tested with TensorFlow 2.10.
    • Add support for Apple Silicon Macs (arm64) via the tensorflow-macos dependency. (#1850)
    • New implementation of GPR and SGPR posterior objects. This primarily improves numerical stability. (#1960)
      • For the GPR this is also a speed improvement when using a GPU.
      • For the SGPR this is a mixed bag, performance-wise.
    • Improved checking and error reporting for the models than do not support full_cov and full_output_cov.
    • Documentation improvements:
      • Improved MCMC notebook.
      • Deleted notebooks that had no contents.
      • Fixed some broken formatting.

    Thanks to our Contributors

    This release contains contributions from:

    jesnie, corwinpro, st--, vdutor

    Source code(tar.gz)
    Source code(zip)
  • v2.5.2(May 10, 2022)

    Release 2.5.2

    This release fixes a performance regression introduced in 2.5.0. 2.5.0 used features of Python that tensorfow < 2.9.0 do not know how to compile, which negatively impacted performance.

    Bug Fixes and Other Changes

    • Fixed some bugs that prevented TensorFlow compilation and had negative performance impact. (#1882)
    • Various improvements to documentation. (#1875, #1866, #1877, #1879)

    Thanks to our Contributors

    This release contains contributions from:

    jesnie

    Source code(tar.gz)
    Source code(zip)
  • v2.5.1(Apr 28, 2022)

    Release 2.5.1

    Fix problem with release process of 2.5.0.

    Bug Fixes and Other Changes

    • Fix bug in release process.

    Thanks to our Contributors

    This release contains contributions from:

    jesnie

    Source code(tar.gz)
    Source code(zip)
  • v2.5.0(Apr 28, 2022)

    Release 2.5.0

    The focus of this release has mostly been bumping the minimally supported versions of Python and TensorFlow; and development of gpflow.experimental.check_shapes.

    Breaking Changes

    • Dropped support for Python 3.6. New minimum version is 3.7. (#1803, #1859)
    • Dropped support for TensorFlow 2.2 and 2.3. New minimum version is 2.4. (#1803)
    • Removed sub-package gpflow.utilities.utilities. It was scheduled for deletion in 2.3.0. Use gpflow.utilities instead. (#1804)
    • Removed method Likelihood.predict_density, which has been deprecated since March 24, 2020. (#1804)
    • Removed property ScalarLikelihood.num_gauss_hermite_points, which has been deprecated since September 30, 2020. (#1804)

    Known Caveats

    • Further improvements to type hints - this may reveal new problems in your code-base if you use a type checker, such as mypy. (#1795, #1799, #1802, #1812, #1814, #1816)

    Major Features and Improvements

    • Significant work on gpflow.experimental.check_shapes.

      • Support anonymous dimensions. (#1796)
      • Add a hook to let the user register shapes for custom types. (#1798)
      • Support Optional values. (#1797)
      • Make it configurable. (#1810)
      • Add accesors for setting/getting previously applied checks. (#1815)
      • Much improved error messages. (#1822)
      • Add support for user notes on shapes. (#1836)
      • Support checking all elements of collections. (#1840)
      • Enable stand-alone shape checking, without using a decorator. (#1845)
      • Support for broadcasts. (#1849)
      • Add support for checking the shapes of intermediate computations. (#1853)
      • Support conditional shapes. (#1855)
    • Significant speed-up of the GPR posterior objects. (#1809, #1811)

    • Significant improvements to documentation. Note the new home page: https://gpflow.github.io/GPflow/index.html (#1828, #1829, #1830, #1831, #1833, #1841, #1842, #1856, #1857)

    Bug Fixes and Other Changes

    • Minor improvement to code clarity (variable scoping) in SVGP model. (#1800)
    • Improving mathematical formatting in docs (SGPR derivations). (#1806)
    • Allow anisotropic kernels to have negative length-scales. (#1843)

    Thanks to our Contributors

    This release contains contributions from:

    ltiao, uri.granta, frgsimpson, st--, jesnie

    Source code(tar.gz)
    Source code(zip)
  • v2.4.0(Mar 1, 2022)

    Release 2.4.0

    This release mostly focuses on make posterior objects useful for Bayesian Optimisation. It also adds a new experimetal sub-package, with a tool for annotating tensor shapes.

    Breaking Changes

    • Slight change to the API of custom posterior objects. gpflow.posteriors.AbstractPosterior._precompute no longer must return an alpha and an Qinv - instead it returns any arbitrary tuple of PrecomputedValues. Correspondingly gpflow.posteriors.AbstractPosterior._conditional_with_precompute should no longer try to access self.alpha and self.Qinv, but instead is passed the tuple of tensors returned by _precompute, as a parameter. (#1763, #1767)

    • Slight change to the API of inducing points. You should no longer override gpflow.inducing_variables.InducingVariables.__len__. Override gpflow.inducing_variables.InducingVariables.num_inducing instead. num_inducing should return a tf.Tensor which is consistent with previous behaviour, although the type previously was annotated as int. __len__ has been deprecated. (#1766, #1792)

    Known Caveats

    • Type hints have been added in several places - this may reveal new problems in your code-base if you use a type checker, such as mypy. (#1766, #1769, #1771, #1773, #1775, #1777, #1780, #1783, #1787, #1789)

    Major Features and Improvements

    • Add new posterior class to enable faster predictions from the VGP model. (#1761)

    • VGP class bug-fixed to work with variable-sized data. Note you can use gpflow.models.vgp.update_vgp_data to ensure variational parameters are updated sanely. (#1774).

    • All posterior classes bug-fixed to work with variable data sizes, for Bayesian Optimisation. (#1767)

    • Added experimental sub-package for features that are still under developmet.

      • Added gpflow.experimental.check_shapes for checking tensor shapes. (#1760, #1768, #1782, #1785, #1788)

    Bug Fixes and Other Changes

    • Make dataclasses dependency conditional at install time. (#1759)
    • Simplify calculations of some predict_f. (#1755)

    Thanks to our Contributors

    This release contains contributions from:

    jesnie, tmct, joacorapela

    Source code(tar.gz)
    Source code(zip)
  • v2.3.1(Jan 20, 2022)

    Release 2.3.1

    This is a bug-fix release, primarily for the GPR posterior object.

    Bug Fixes and Other Changes

    • GPR posterior

      • Fix the calculation in the GPR posterior object (#1734).
      • Fixes leading dimension issues with GPRPosterior._conditional_with_precompute() (#1747).
    • Make gpflow.optimizers.Scipy able to handle unused / unconnected variables. (#1745).

    • Build

      • Fixed broken CircleCi build (#1738).
      • Update CircleCi build to use next-gen Docker images (#1740).
      • Fixed broken triggering of docs generation (#1744).
      • Make all slow tests depend on fast tests (#1743).
      • Make make dev-install also install the test requirements (#1737).
    • Documentation

      • Fixed broken link in README.md (#1736).
      • Fix broken build of cglb.ipynb (#1742).
      • Add explanation of how to run notebooks locally (#1729).
      • Fix formatting in notebook on Heteroskedastic Likelihood (#1727).
      • Fix broken link in introduction (#1718).
    • Test suite

      • Amends test_gpr_posterior.py so it will cover leading dimension uses.

    Thanks to our Contributors

    This release contains contributions from:

    st--, jesnie, johnamcleod, Andrew878

    Source code(tar.gz)
    Source code(zip)
  • v2.3.0(Oct 26, 2021)

    Major Features and Improvements

    • Refactor posterior base class to support other model types. (#1695)
    • Add new posterior class to enable faster predictions from the GPR/SGPR models. (#1696, #1711)
    • Construct Parameters from other Parameters and retain properties. (#1699)
    • Add CGLB model (#1706)

    Bug Fixes and Other Changes

    • Fix unit test failure when using TensorFlow 2.5.0 (#1684)
    • Upgrade black formatter to version 20.8b1 (#1694)
    • Remove erroneous DeprecationWarnings (#1693)
    • Fix SGPR derivation (#1688)
    • Fix tests which fail with TensorFlow 2.6.0 (#1714)

    Thanks to our Contributors

    This release contains contributions from:

    johnamcleod, st--, Andrew878, tadejkrivec, awav, avullo

    Source code(tar.gz)
    Source code(zip)
  • v2.2.1(Apr 29, 2021)

  • v2.2.0(Apr 27, 2021)

    Release 2.2.0

    The main focus of this release is the new "Posterior" object introduced by PR #1636, which allows for a significant speed-up of post-training predictions with the SVGP model (partially resolving #1599).

    • For end-users, by default nothing changes; see Breaking Changes below if you have written your own implementations of gpflow.conditionals.conditional.
    • After training an SVGP model, you can call model.posterior() to obtain a Posterior object that precomputes all quantities not depending on the test inputs (e.g. Choleskty of Kuu), and provides a posterior.predict_f() method that reuses these cached quantities. model.predict_f() computes exactly the same quantities as before and does not give any speed-up.
    • gpflow.conditionals.conditional() forwards to the same "fused" code-path as before.

    Breaking Changes

    • gpflow.conditionals.conditional.register is deprecated and should not be called outside of the GPflow core code. If you have written your own implementations of gpflow.conditionals.conditional(), you have two options to use your code with GPflow 2.2:
      1. Temporary work-around: Instead of gpflow.models.SVGP, use the backwards-compatible gpflow.models.svgp.SVGP_deprecated.
      2. Convert your conditional() implementation into a subclass of gpflow.posteriors.AbstractPosterior, and register get_posterior_class() instead (see the "Variational Fourier Features" notebook for an example).

    Known Caveats

    • The Posterior object is currently only available for the SVGP model. We would like to extend this to the other models such as GPR, SGPR, or VGP, but this effort is beyond what we can currently provide. If you would be willing to contribute to those efforts, please get in touch!
    • The Posterior object does not currently provide the GPModel convenience functions such as predict_f_samples, predict_y, predict_log_density. Again, if you're willing to contribute, get in touch!

    Thanks to our Contributors

    This release contains contributions from:

    @stefanosele, @johnamcleod, @st--

    Source code(tar.gz)
    Source code(zip)
  • v2.1.5(Apr 15, 2021)

    Known Caveats

    • GPflow requires TensorFlow >= 2.2.

    Deprecations

    • The gpflow.utilities.utilities submodule has been deprecated and will be removed in GPflow 2.3. User code should access functions directly through gpflow.utilities instead (#1650).

    Major Features and Improvements

    • Improves compatibility between monitoring API and Scipy optimizer (#1642).
    • Adds _add_noise_cov method to GPR model class to make it more easily extensible (#1645).

    Bug Fixes

    • Fixes a bug in ModelToTensorBoard (#1619) when max_size=-1 (#1619)

    • Fixes a dynamic shape issue in the quadrature code (#1626).

    • Fixes #1651, a bug in fully_correlated_conditional_repeat (#1652).

    • Fixes #1653, a bug in the "fallback" code path for multioutput Kuf (#1654).

    • Fixes a bug in the un-whitened code path for the fully correlated conditional function (#1662).

    • Fixes a bug in independent_interdomain_conditional (#1663).

    • Fixes an issue with the gpflow.config API documentation (#1664).

    • Test suite

      • Fixes the test suite for TensorFlow 2.4 / TFP 0.12 (#1625).
      • Fixes mypy call (#1637).
      • Fixes a bug in test_method_equivalence.py (#1649).

    Thanks to our Contributors

    This release contains contributions from:

    johnamcleod, st--, vatsalaggarwal, sam-willis, vdutor

    Source code(tar.gz)
    Source code(zip)
  • v2.1.4(Dec 1, 2020)

    Improvements

    • Replace len(inducing_variable) with inducing_variable.num inducing property (#1594). Adds support for inducing variables with dynamically changing shape (compatibility with tf.function). (Resolves #1578.)

    • HeteroskedasticTFPConditional should construct tensors at class-construction, not at module-import time (#1598).

    Source code(tar.gz)
    Source code(zip)
  • v2.1.3(Oct 7, 2020)

    Improvements

    • kullback_leiblers.gauss_kl: when running TensorFlow >= 2.2, we avoid a tf.tile() that was a workaround for broadcasting issues in TensorFlow <= 2.1; the workaround is now only executed when running in TensorFlow 2.1 (#1585).

    Bugfixes

    • model.predict_y/predict_log_density now explicitly raise NotImplementedError when passed full_(output_)cov=True (#1582). Actual implementation of the full_(output_)cov=True case remains an open issue (#1461).
    • bugfix for conditionals.multioutput.util.independent_interdomain_conditional (#1583).

    Docs and infrastructure

    • CI/infrastructure updates (#1580, #1581, #1587, #1592).
    • heteroskedastic notebook cleaned up, links added (#1579).
    • intro_to_gpflow2 notebook updated to account for saved_model now working (#1586).
    Source code(tar.gz)
    Source code(zip)
  • v2.1.2(Sep 30, 2020)

    New features and improvements

    • Likelihoods now use the new Quadrature class (#1566, #1571)
    • Multi-latent likelihoods for heteroskedastic models (#1559)
    • isort for consistent imports (#1572)

    Bugfixes

    • Fix-up to our workaround for https://github.com/tensorflow/probability/issues/547 for tensorflow-probability==0.11.1 (#1574)
    • Add missing packaging dependency (#1568)

    Deprecations

    • You should no longer use likelihood.num_gauss_hermite_points. Interact with its quadrature attribute instead.
    Source code(tar.gz)
    Source code(zip)
  • v2.1.1(Sep 14, 2020)

    This patch release restores full compatibility with TensorFlow 2.3.

    Known installation issue: if import gpflow gives you an ImportError, please install the "packaging" package manually (pip install packaging). Fixed by #1568, which will be included in the next patch release (GPflow 2.1.2).

    Known bug: The deepcopy workaround for https://github.com/tensorflow/probability/issues/547 turned out not to be compatible with tensorflow-probability==0.11.1 which was released on 29 September 2020 on PyPI. We fixed this in #1574, which will be included in the following patch release (GPflow 2.1.2).

    Improvements

    • Enable compatibility with TensorFlow 2.3 and tensorflow_probability 0.11.0 (#1553)
    • Recommend pip --use-feature=2020-resolver for installation
    • Ensure all gpflow code files contain copyright & license note (#1480)

    Bugfixes

    • Fix SGPR upper bound for non zero mean functions (#1560)
    • Fix broadcasting for constant and zero mean functions (#1550)
    Source code(tar.gz)
    Source code(zip)
  • v2.1.0(Aug 27, 2020)

    Release 2.1.0

    Dependency constraints: To work around incompatibilities in our TensorFlow dependencies, GPflow 2.1.0 has the following new version requirements (#1522, #1537, #1551): tensorflow<2.3, tensorflow_probability>=0.10.1,<0.11, cloudpickle==1.3.0. We have restored compatibility with TensorFlow 2.3 and tensorflow_probability>=0.11 in the GPflow 2.1.1 patch release.

    Key improvements

    • gpflow.Parameter finally plays well with TensorFlow's saving; instead of pretending to be a tf.Variable, we're now building on top of tfp.util.TransformedVariable (#1518) - see backwards incompatibilities below
    • Gauss-Hermite quadrature code was heavily refactored. The old function gpflow.quadrature.ndiagquad has been replaced by the gpflow.quadrature.NDiagGHQuadrature class - we strongly recommend upgrading (note that the dimensions of quadrature points have been moved from the end to the start for better broadcasting); ndiagquad will eventually be deprecated (#1505, #1542).

    Minor changes

    • Add base_conditional_with_lm function that gets passed cholesky(K) instead of computing it explicitly (#1528)
    • Update to use stock rather than custom Docker image for CI (#1545)
    • Check dependency versions are valid on CI (#1536)
    • Docs: Readme updated to include new project that is using GPflow (#1530)

    Bug Fixes

    • Restore pytest-xdist compatibility for parallelizing local tests (#1541)
    • Fix for quadrature failure mode when autograph was set to False (#1548)
    • Fix formatting in docs (intro.md) and restore link removed by #1498 (#1520)
    • Fix bug in varying_noise notebook (#1526)
    • Fix: separate_independent_conditional now correctly handles q_sqrt=None. (#1533)

    Backwards incompatibilities

    • gpflow.Parameter is no longer pretending to be a tf.Variable, but instead subclasses tensorflow_probability's TransformedVariable. Several of the Variable-specific attributes/methods disappeared. Specifically, if you had previously called a Parameter's value() or read_value() methods, you should simply remove these.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.5(Jun 11, 2020)

    Improvements

    • gpflow.utilities.set_trainable() can now handle an Iterable of tf.Modules in one go (#1486)
    • Clarification: the NaturalGradient optimizer does not handle the diagonal (q_diag=True) case; emphasised this in the documentation and added shape checks (#1489)
    • Added type hints (#1487)
    • Potentially backwards-incompatible: gpflow.optimizers.Scipy: the callable passed in as step_callback is now passed the arguments (step, variables, values) by position, no longer by name (#1454)
    • Minor documentation improvements (#1488, #1492, #1498, #1503)

    Bug fixes

    • multi-output combination kernels now support the presliced argument (#1495)
    • GPR can now be used with tf.Variables holding the data (#1490)
    • fixed indentation error in Makefile (#1482)
    Source code(tar.gz)
    Source code(zip)
  • v2.0.4(May 20, 2020)

    Improvements

    • Add type hints to gpflow.base module (https://github.com/GPflow/GPflow/pull/1427)
    • Enable mypy checking on CI (https://github.com/GPflow/GPflow/pull/1471)

    Bugfixes

    • TensorFlow Probability 0.10 introduces a new _parameters attribute in tfp.bijectors.Bijector classes. That attribute contains a self-reference that broke module traversing. In this fix, module traverse ignores _parameters attribute.
    • Handcrafted utilities.deepcopy was improved using memorization argument of python's copy.deepcopy method. The change prevents double copying of objects. (https://github.com/GPflow/GPflow/pull/1476)
    Source code(tar.gz)
    Source code(zip)
  • v2.0.3(May 13, 2020)

  • v2.0.2(May 7, 2020)

    • Refactor of Natural Gradient optimizer implementation to be more easily extendable and to more efficiently handle updating several variational distributions simultaneously (#1443)
    • Change function signature of sample_mvn to take full_cov boolean instead of cov_structure string (#1448)
    • Fixes bug in Changepoint kernel evaluation (#1446)
    • Fixes #1439 (TensorFlow error when using larger datasets with tf.function compilation) in GPflow models that store data by converting it to tensors first, introduces data_input_to_tensor utility function (#1452, #1458)
    • Workaround for bug in TensorFlow <= 2.2.0rc3 in the leading_transpose utility function (#1442)
    • Improved representation of GPflow objects in IPython shell and Jupyter notebooks (#1453)
    • Minor cleanups and added type hints (#1421, #1438, #1441, #1450)
    Source code(tar.gz)
    Source code(zip)
  • v2.0.1(Apr 15, 2020)

    Release notes:

    • Improve structure of likelihoods subdirectory (#1416)
    • Update README.md (#1401) and GPflow 2 upgrade guide (#1414)
    • Improved handling of invalid values for constrained Parameters (#1408)
    • Improvements on types/function annotations (#1406, #1420)
    • Documentation improvements (metalearning with GPs: #1382, coregionalization notebook: #1402, MCMC notebook: #1410, intro to gpflow with tensorflow 2: #1413)
    • Minor documentation fixes (#1429, #1430, #1433)
    • Fix: move matplotlib import inside ImageToTensorBoard (#1399)
    • Fix: tf.function compilation of ndiagquad (#1418)
    • Fix: cache tensorboard file writers and re-use them (#1424)
    Source code(tar.gz)
    Source code(zip)
  • v2.0.0(Mar 31, 2020)

    GPflow 2.0 supports TF2.1 and does not support older versions TF1.x and TF2.0. GPflow stops accepting bug reports and PRs to the GPflow develop-1.0 branch. Please check the “GPflow 2 upgrade guide” and “Intro to GPflow with TensorFlow 2” notebooks in the documentation.

    • Parameter behaves as a tf.Tensor
    • Parameter has an option to set a prior on constrained or unconstrained space (#1177)
    • Parameter uses bijectors from TensorFlow Probability as a transformation. A positive bijector is provided in gpflow.utilities that can be configured to exp or softplus and has a variable lower bound.
    • Parameter doesn't have the setter for the trainable attribute. New utility function set_trainable added as a replacement.
    • Add GPflow config module for global settings: default float and integer types, default jitter, default positive transformation, minimal positive value, default printing format. Default settings can be changed using environment variables. The gpflowrc file is not supported anymore (#1148, #1267, #1135)
    • Add GPflow utility module. This module has many useful functions: custom operations like leading_transpose and broadcasting_elementwise, utility functions like customized deepcopy for tf.Module objects, freeze method that turns mutable models to immutable copies, to_default_float and many more (#1293)
    • Likelihoods have a more tightly defined interface, increasing shape robustness and making it easier to incorporate non-scalar likelihoods (#1334)
    • Add new Scipy optimizer that mimics standard scipy.minimize interface.
    • Remove custom GPflow optimizers and HMC implementation. A user can exploit standard TensorFlow optimizers and TensorFlow Probability MCMC sampling methods accordingly (#1100)
    • Add new monitor implementation (#1344)
    • Refactor stationary kernels (#1088)
    • Add Convolutional kernel (#996, #1034)
    • Add generalized Periodic kernel (#1158)
    • Add ChangePoint kernel (#1094)
    • Conditional calculations only take the lower triangle parts of q_sqrt into inconsideration (#1223)
    • Numerous improvements to notebooks and documentation:
      • Raw Jupyter notebook files replaced with jupytext python files (#1239)
      • Documentation generation in a separate repo (#1275)
    • Improved Bayesian model classes hierarchy reflecting the difference between data handling of models (#1276)
    • log_prior was renamed to log_prior_density (#1329)
    • predict_density was renamed to predict_log_density
    • full in the kernel.call was renamed to full_cov (#1319)
    • num_latent was renamed to num_latent_gps (#1310)

    Thanks a lot to everyone who made GPflow 2.0 happen:

    Artem Artemev, @awav S.T. John, @st-- Vincent Dutordoir, @vdutor Mark van der Wilk, @markvdw James Hensman, @jameshensman Eric Hambro, @condnsdmatters Joel Berkeley, @joelberkeley-pio Alexis Boukouvalas, @alexisboukouvalas Sergio Pascual Diaz, @spascual Vincent Adam, @thevincentadam Fergus Simpson, @frgsimpson @ilia-kats @marcoadurno Joe Hall, @jahall Tadej Krivec, @tadejkrivec (SGPR theory derivation fix) Alexandra Hayes, @akhayes; James Leedham, (copyediting on notebooks) Nicolas Durrande, @NicolasDurrande; Felix Leibfried, @fleibfried; Jordi Graumo, @jordigraumo (porting notebooks to gpflow 2.0) Neil Ferguson, @pio-neil (GPflow 2 upgrade guide)

    Source code(tar.gz)
    Source code(zip)
  • 2.0.0-rc1(Nov 14, 2019)

    GPflow works with TensorFlow 2.0!

    Features:

    • No static graphs, pythonic way of writing the code
    • Refactored GPflow configuration
    • Updated model printing
    • Fast multiple assign
    • Convolutional kernel
    • ChangePoint kernel
    Source code(tar.gz)
    Source code(zip)
  • v1.5.1(Nov 3, 2019)

    Features and improvements:

    • Copyedits in documentation and notebooks
    • Make likelihood quadrature tests comprehensive (#1081)

    Bugfixes:

    • Fix an issue with graph pollution in the monitor task ImageToTensorboardTask (#1041)
    Source code(tar.gz)
    Source code(zip)
  • v1.5.0(Aug 13, 2019)

    Features and improvements:

    • Reusable ScipyOptimizer (#1011)
    • Documentation copyedits (#1019, #1013-1017)
    • Convolutional kernel (#996)
    • Saver works with file-like objects as well as path names (#912)

    Bugfixes:

    • The misc.initialize_variables() does not keep adding ops to the graph (#1001)
    • Variables restored in monitoring notebook (#1003)
    • Support TensorFlow 1.14.0 (#997)
    • Shape checks for multivariate_normal (#964)
    Source code(tar.gz)
    Source code(zip)
  • v1.4.1(Jun 17, 2019)

    Features, bugfixes and improvements:

    • Minor performance improvement: pass cholesky to the gauss_kl (#933)
    • Documentation revamp:
      • Organize documentation and notebooks (#924)
      • GPflow glossary document (#923)
      • Fix AutoBuild metaclass for method signatures (#907)
    • CircleCI instead of TravisCI (#892)
    • Broadcasting conditionals via tf.broadcast_to (#895)
    • Support TensorFlow >= 1.12 (#877)
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Oct 19, 2018)

    Main features and improvements

    • Cleaning up stationary kernel implementations: now defined in terms of K_r or K_r2. (#827)
    • Support broadcasting over arbitrarily many leading dimensions for kernels and conditional. (#829)
    • Analytic expectation of the cross-covariance between different RBF kernels. (#754)
    • New MixedKernelSeparateMof feature class for multi-output GPs. (#830)
    • The sample_conditional returns mean and var as well as samples, and can generate more than one sample. (#836)
    • Support monitoring with ScipyOptimizer. (#856)

    Bug fixes

    • Fix bug in ndiag_mc for multi-dimensional kwargs. (#813)
    • Fix parameter.trainable to be a property. (#814)
    • Remove references to six module. (#816)
    • Fix tf.control_dependencies in likelihoods. (#821)
    • Fix active_dims for slice type. (#840)
    Source code(tar.gz)
    Source code(zip)
Owner
GPflow
GPflow
A python package which can be pip installed to perform statistics and visualize binomial and gaussian distributions of the dataset

GBiStat package A python package to assist programmers with data analysis. This package could be used to plot : Binomial Distribution of the dataset p

Rishikesh S 4 Oct 17, 2022
Probabilistic reasoning and statistical analysis in TensorFlow

TensorFlow Probability TensorFlow Probability is a library for probabilistic reasoning and statistical analysis in TensorFlow. As part of the TensorFl

null 3.8k Jan 5, 2023
A probabilistic programming language in TensorFlow. Deep generative models, variational inference.

Edward is a Python library for probabilistic modeling, inference, and criticism. It is a testbed for fast experimentation and research with probabilis

Blei Lab 4.7k Jan 9, 2023
A probabilistic programming library for Bayesian deep learning, generative models, based on Tensorflow

ZhuSuan is a Python probabilistic programming library for Bayesian deep learning, which conjoins the complimentary advantages of Bayesian methods and

Tsinghua Machine Learning Group 2.2k Dec 28, 2022
Gaussian processes in TensorFlow

Website | Documentation (release) | Documentation (develop) | Glossary Table of Contents What does GPflow do? Installation Getting Started with GPflow

GPflow 1.7k Jan 6, 2023
A Tensorflow based library for Time Series Modelling with Gaussian Processes

Markovflow Documentation | Tutorials | API reference | Slack What does Markovflow do? Markovflow is a Python library for time-series analysis via prob

Secondmind Labs 24 Dec 12, 2022
A highly efficient and modular implementation of Gaussian Processes in PyTorch

GPyTorch GPyTorch is a Gaussian process library implemented using PyTorch. GPyTorch is designed for creating scalable, flexible, and modular Gaussian

null 3k Jan 2, 2023
A Python implementation of global optimization with gaussian processes.

Bayesian Optimization Pure Python implementation of bayesian global optimization with gaussian processes. PyPI (pip): $ pip install bayesian-optimizat

fernando 6.5k Jan 2, 2023
A highly efficient and modular implementation of Gaussian Processes in PyTorch

GPyTorch GPyTorch is a Gaussian process library implemented using PyTorch. GPyTorch is designed for creating scalable, flexible, and modular Gaussian

null 3k Jan 2, 2023
Supplementary code for the AISTATS 2021 paper "Matern Gaussian Processes on Graphs".

Matern Gaussian Processes on Graphs This repo provides an extension for gpflow with Matérn kernels, inducing variables and trainable models implemente

null 41 Dec 17, 2022
Bayesian optimization based on Gaussian processes (BO-GP) for CFD simulations.

BO-GP Bayesian optimization based on Gaussian processes (BO-GP) for CFD simulations. The BO-GP codes are developed using GPy and GPyOpt. The optimizer

KTH Mechanics 8 Mar 31, 2022
A bare-bones TensorFlow framework for Bayesian deep learning and Gaussian process approximation

Aboleth A bare-bones TensorFlow framework for Bayesian deep learning and Gaussian process approximation [1] with stochastic gradient variational Bayes

Gradient Institute 127 Dec 12, 2022
HiddenMarkovModel implements hidden Markov models with Gaussian mixtures as distributions on top of TensorFlow

Class HiddenMarkovModel HiddenMarkovModel implements hidden Markov models with Gaussian mixtures as distributions on top of TensorFlow 2.0 Installatio

Susara Thenuwara 2 Nov 3, 2021
Debugger capable of attaching to and injecting code into python processes.

DISCLAIMER: This is not an official google project, this is just something I wrote while at Google. Pyringe What this is Pyringe is a python debugger

Google 1.6k Dec 15, 2022
Inject code into running Python processes

pyrasite Tools for injecting arbitrary code into running Python processes. homepage: http://pyrasite.com documentation: http://pyrasite.rtfd.org downl

Luke Macken 2.7k Jan 8, 2023
A colony of interacting processes

NColony Infrastructure for running "colonies" of processes. Hacking $ tox Should DTRT -- if it passes, it means unit tests are passing, and 100% cover

null 23 Apr 4, 2022
Official code for the ICLR 2021 paper Neural ODE Processes

Neural ODE Processes Official code for the paper Neural ODE Processes (ICLR 2021). Abstract Neural Ordinary Differential Equations (NODEs) use a neura

Cristian Bodnar 50 Oct 28, 2022
Socorro is the Mozilla crash ingestion pipeline. It accepts and processes Breakpad-style crash reports. It provides analysis tools.

Socorro Socorro is a Mozilla-centric ingestion pipeline and analysis tools for crash reports using the Breakpad libraries. Support This is a Mozilla-s

Mozilla Services 552 Dec 19, 2022
Implementation of "Fast and Flexible Temporal Point Processes with Triangular Maps" (Oral @ NeurIPS 2020)

Fast and Flexible Temporal Point Processes with Triangular Maps This repository includes a reference implementation of the algorithms described in "Fa

Oleksandr Shchur 20 Dec 2, 2022