TensorFlow-based neural network library

Overview

Sonnet

Sonnet

Documentation | Examples

Sonnet is a library built on top of TensorFlow 2 designed to provide simple, composable abstractions for machine learning research.

Introduction

Sonnet has been designed and built by researchers at DeepMind. It can be used to construct neural networks for many different purposes (un/supervised learning, reinforcement learning, ...). We find it is a successful abstraction for our organization, you might too!

More specifically, Sonnet provides a simple but powerful programming model centered around a single concept: snt.Module. Modules can hold references to parameters, other modules and methods that apply some function on the user input. Sonnet ships with many predefined modules (e.g. snt.Linear, snt.Conv2D, snt.BatchNorm) and some predefined networks of modules (e.g. snt.nets.MLP) but users are also encouraged to build their own modules.

Unlike many frameworks Sonnet is extremely unopinionated about how you will use your modules. Modules are designed to be self contained and entirely decoupled from one another. Sonnet does not ship with a training framework and users are encouraged to build their own or adopt those built by others.

Sonnet is also designed to be simple to understand, our code is (hopefully!) clear and focussed. Where we have picked defaults (e.g. defaults for initial parameter values) we try to point out why.

Getting Started

Examples

The easiest way to try Sonnet is to use Google Colab which offers a free Python notebook attached to a GPU or TPU.

Installation

To get started install TensorFlow 2.0 and Sonnet 2:

$ pip install tensorflow-gpu tensorflow-probability
$ pip install dm-sonnet

You can run the following to verify things installed correctly:

import tensorflow as tf
import sonnet as snt

print("TensorFlow version {}".format(tf.__version__))
print("Sonnet version {}".format(snt.__version__))

Using existing modules

Sonnet ships with a number of built in modules that you can trivially use. For example to define an MLP we can use the snt.Sequential module to call a sequence of modules, passing the output of a given module as the input for the next module. We can use snt.Linear and tf.nn.relu to actually define our computation:

mlp = snt.Sequential([
    snt.Linear(1024),
    tf.nn.relu,
    snt.Linear(10),
])

To use our module we need to "call" it. The Sequential module (and most modules) define a __call__ method that means you can call them by name:

logits = mlp(tf.random.normal([batch_size, input_size]))

It is also very common to request all the parameters for your module. Most modules in Sonnet create their parameters the first time they are called with some input (since in most cases the shape of the parameters is a function of the input). Sonnet modules provide two properties for accessing parameters.

The variables property returns all tf.Variables that are referenced by the given module:

all_variables = mlp.variables

It is worth noting that tf.Variables are not just used for parameters of your model. For example they are used to hold state in metrics used in snt.BatchNorm. In most cases users retrieve the module variables to pass them to an optimizer to be updated. In this case non-trainable variables should typically not be in that list as they are updated via a different mechanism. TensorFlow has a built in mechanism to mark variables as "trainable" (parameters of your model) vs. non-trainable (other variables). Sonnet provides a mechanism to gather all trainable variables from your module which is probably what you want to pass to an optimizer:

model_parameters = mlp.trainable_variables

Building your own module

Sonnet strongly encourages users to subclass snt.Module to define their own modules. Let's start by creating a simple Linear layer called MyLinear:

class MyLinear(snt.Module):

  def __init__(self, output_size, name=None):
    super(MyLinear, self).__init__(name=name)
    self.output_size = output_size

  @snt.once
  def _initialize(self, x):
    initial_w = tf.random.normal([x.shape[1], self.output_size])
    self.w = tf.Variable(initial_w, name="w")
    self.b = tf.Variable(tf.zeros([self.output_size]), name="b")

  def __call__(self, x):
    self._initialize(x)
    return tf.matmul(x, self.w) + self.b

Using this module is trivial:

mod = MyLinear(32)
mod(tf.ones([batch_size, input_size]))

By subclassing snt.Module you get many nice properties for free. For example a default implementation of __repr__ which shows constructor arguments (very useful for debugging and introspection):

>>> print(repr(mod))
MyLinear(output_size=10)

You also get the variables and trainable_variables properties:

>>> mod.variables
(<tf.Variable 'my_linear/b:0' shape=(10,) ...)>,
 <tf.Variable 'my_linear/w:0' shape=(1, 10) ...)>)

You may notice the my_linear prefix on the variables above. This is because Sonnet modules also enter the modules name scope whenever methods are called. By entering the module name scope we provide a much more useful graph for tools like TensorBoard to consume (e.g. all operations that occur inside my_linear will be in a group called my_linear).

Additionally your module will now support TensorFlow checkpointing and saved model which are advanced features covered later.

Serialization

Sonnet supports multiple serialization formats. The simplest format we support is Python's pickle, and all built in modules are tested to make sure they can be saved/loaded via pickle in the same Python process. In general we discourage the use of pickle, it is not well supported by many parts of TensorFlow and in our experience can be quite brittle.

TensorFlow Checkpointing

Reference: https://www.tensorflow.org/alpha/guide/checkpoints

TensorFlow checkpointing can be used to save the value of parameters periodically during training. This can be useful to save the progress of training in case your program crashes or is stopped. Sonnet is designed to work cleanly with TensorFlow checkpointing:

checkpoint_root = "/tmp/checkpoints"
checkpoint_name = "example"
save_prefix = os.path.join(checkpoint_root, checkpoint_name)

my_module = create_my_sonnet_module()  # Can be anything extending snt.Module.

# A `Checkpoint` object manages checkpointing of the TensorFlow state associated
# with the objects passed to it's constructor. Note that Checkpoint supports
# restore on create, meaning that the variables of `my_module` do **not** need
# to be created before you restore from a checkpoint (their value will be
# restored when they are created).
checkpoint = tf.train.Checkpoint(module=my_module)

# Most training scripts will want to restore from a checkpoint if one exists. This
# would be the case if you interrupted your training (e.g. to use your GPU for
# something else, or in a cloud environment if your instance is preempted).
latest = tf.train.latest_checkpoint(checkpoint_root)
if latest is not None:
  checkpoint.restore(latest)

for step_num in range(num_steps):
  train(my_module)

  # During training we will occasionally save the values of weights. Note that
  # this is a blocking call and can be slow (typically we are writing to the
  # slowest storage on the machine). If you have a more reliable setup it might be
  # appropriate to save less frequently.
  if step_num and not step_num % 1000:
    checkpoint.save(save_prefix)

# Make sure to save your final values!!
checkpoint.save(save_prefix)

TensorFlow Saved Model

Reference: https://www.tensorflow.org/alpha/guide/saved_model

TensorFlow saved models can be used to save a copy of your network that is decoupled from the Python source for it. This is enabled by saving a TensorFlow graph describing the computation and a checkpoint containing the value of weights.

The first thing to do in order to create a saved model is to create a snt.Module that you want to save:

my_module = snt.nets.MLP([1024, 1024, 10])
my_module(tf.ones([1, input_size]))

Next, we need to create another module describing the specific parts of our model that we want to export. We advise doing this (rather than modifying the original model in-place) so you have fine grained control over what is actually exported. This is typically important to avoid creating very large saved models, and such that you only share the parts of your model you want to (e.g. you only want to share the generator for a GAN but keep the discriminator private).

@tf.function(input_signature=[tf.TensorSpec([None, input_size])])
def inference(x):
  return my_module(x)

to_save = snt.Module()
to_save.inference = inference
to_save.all_variables = list(my_module.variables)
tf.saved_model.save(to_save, "/tmp/example_saved_model")

We now have a saved model in the /tmp/example_saved_model folder:

$ ls -lh /tmp/example_saved_model
total 24K
drwxrwsr-t 2 tomhennigan 154432098 4.0K Apr 28 00:14 assets
-rw-rw-r-- 1 tomhennigan 154432098  14K Apr 28 00:15 saved_model.pb
drwxrwsr-t 2 tomhennigan 154432098 4.0K Apr 28 00:15 variables

Loading this model is simple and can be done on a different machine without any of the Python code that built the saved model:

loaded = tf.saved_model.load("/tmp/example_saved_model")

# Use the inference method. Note this doesn't run the Python code from `to_save`
# but instead uses the TensorFlow Graph that is part of the saved model.
loaded.inference(tf.ones([1, input_size]))

# The all_variables property can be used to retrieve the restored variables.
assert len(loaded.all_variables) > 0

Note that the loaded object is not a Sonnet module, it is a container object that has the specific methods (e.g. inference) and properties (e.g. all_variables) that we added in the previous block.

Distributed training

Example: https://github.com/deepmind/sonnet/blob/v2/examples/distributed_cifar10.ipynb

Sonnet has support for distributed training using custom TensorFlow distribution strategies.

A key difference between Sonnet and distributed training using tf.keras is that Sonnet modules and optimizers do not behave differently when run under distribution strategies (e.g. we do not average your gradients or sync your batch norm stats). We believe that users should be in full control of these aspects of their training and they should not be baked into the library. The trade off here is that you need to implement these features in your training script (typically this is just 2 lines of code to all reduce your gradients before applying your optimizer) or swap in modules that are explicitly distribution aware (e.g. snt.distribute.CrossReplicaBatchNorm).

Our distributed Cifar-10 example walks through doing multi-GPU training with Sonnet.

Comments
  • internal compiler error: in tsubst_copy, at cp/pt.c:13970 using gcc 6.2

    internal compiler error: in tsubst_copy, at cp/pt.c:13970 using gcc 6.2

    I have been unable to install Sonnet because of a compiler error. I tried to find if this was a bug with gcc and I found similar entries, however updating gcc did not work. This error happened to me using verions 5.3 and 6.2 of gcc (output of the latter below).

    abermea@host:~/Projects/Sonnet/sonnet$ bazel build --config=opt :install
    .
    WARNING: Config values are not defined in any .rc file: opt
    WARNING: /home/abermea/.cache/bazel/_bazel_abermea/ca99c09533717eb94266b31b726808fb/external/org_tensorflow/tensorflow/workspace.bzl:72:5: tf_repo_name was specified to tf_workspace but is no longer used and will be removed in the future.
    INFO: Found 1 target...
    ERROR: /home/abermea/Projects/Sonnet/sonnet/sonnet/cc/kernels/BUILD:19:1: C++ compilation of rule '//sonnet/cc/kernels:resampler_op' failed: Process exited with status 1 [sandboxed].
    sonnet/cc/kernels/resampler_op.cc: In instantiation of 'deepmind::tensorflow::sonnet::functor::ResamplerGrad2DFunctor<Eigen::ThreadPoolDevice, T>::operator()(tensorflow::OpKernelContext*, const CPUDevice&, const T*, const T*, const T*, T*, T*, int, int, int, int, int)::<lambda(int, int)>::<lambda(int, int, int, T)> [with T = double]':
    sonnet/cc/kernels/resampler_op.cc:269:23:   required from 'struct deepmind::tensorflow::sonnet::functor::ResamplerGrad2DFunctor<Eigen::ThreadPoolDevice, T>::operator()(tensorflow::OpKernelContext*, const CPUDevice&, const T*, const T*, const T*, T*, T*, int, int, int, int, int)::<lambda(int, int)> [with T = double]::<lambda(int, int, int, double)>'
    sonnet/cc/kernels/resampler_op.cc:272:9:   required from 'deepmind::tensorflow::sonnet::functor::ResamplerGrad2DFunctor<Eigen::ThreadPoolDevice, T>::operator()(tensorflow::OpKernelContext*, const CPUDevice&, const T*, const T*, const T*, T*, T*, int, int, int, int, int)::<lambda(int, int)> [with T = double]'
    sonnet/cc/kernels/resampler_op.cc:317:38:   required from 'struct deepmind::tensorflow::sonnet::functor::ResamplerGrad2DFunctor<Eigen::ThreadPoolDevice, T>::operator()(tensorflow::OpKernelContext*, const CPUDevice&, const T*, const T*, const T*, T*, T*, int, int, int, int, int) [with T = double; deepmind::tensorflow::sonnet::CPUDevice = Eigen::ThreadPoolDevice]::<lambda(int, int)>'
    sonnet/cc/kernels/resampler_op.cc:338:5:   required from 'void deepmind::tensorflow::sonnet::functor::ResamplerGrad2DFunctor<Eigen::ThreadPoolDevice, T>::operator()(tensorflow::OpKernelContext*, const CPUDevice&, const T*, const T*, const T*, T*, T*, int, int, int, int, int) [with T = double; deepmind::tensorflow::sonnet::CPUDevice = Eigen::ThreadPoolDevice]'
    sonnet/cc/kernels/resampler_op.cc:407:51:   required from 'void deepmind::tensorflow::sonnet::ResamplerGradOp<Device, T>::Compute(tensorflow::OpKernelContext*) [with Device = Eigen::ThreadPoolDevice; T = double]'
    sonnet/cc/kernels/resampler_op.cc:443:1:   required from here
    sonnet/cc/kernels/resampler_op.cc:239:47: internal compiler error: in tsubst_copy, at cp/pt.c:13970
         const int data_batch_stride = data_height * data_width * data_channels;
                                       ~~~~~~~~~~~~^~~~~~~~~~~~
    0x60e858 tsubst_copy
    	../../src/gcc/cp/pt.c:13970
    0x60efb1 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:17067
    0x6102b8 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:16252
    0x6102b8 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:16252
    0x60aa58 tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15876
    0x60bab5 tsubst_init
    	../../src/gcc/cp/pt.c:13916
    0x60e8e6 tsubst_copy
    	../../src/gcc/cp/pt.c:14109
    0x60efb1 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:17067
    0x6102d6 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:16253
    0x6102b8 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:16252
    0x6102b8 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:16252
    0x60f2bf tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:16285
    0x60fa64 tsubst_copy_and_build(tree_node*, tree_node*, int, tree_node*, bool, bool)
    	../../src/gcc/cp/pt.c:16390
    0x60aa58 tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15876
    0x609686 tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15192
    0x60a903 tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15364
    0x609980 tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15344
    0x60a8bc tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15178
    0x60a903 tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15364
    0x60a8bc tsubst_expr(tree_node*, tree_node*, int, tree_node*, bool)
    	../../src/gcc/cp/pt.c:15178
    Please submit a full bug report,
    with preprocessed source if appropriate.
    Please include the complete backtrace with any bug report.
    See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
    Use --strategy=CppCompile=standalone to disable sandboxing for the failing actions.
    Target //:install failed to build
    Use --verbose_failures to see the command lines of failed build steps.
    INFO: Elapsed time: 34.787s, Critical Path: 11.86s
    
    ==========================================
    
    abermea@host:~/Projects/Sonnet/sonnet$ gcc -v
    Using built-in specs.
    COLLECT_GCC=gcc
    COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/6/lto-wrapper
    Target: x86_64-linux-gnu
    Configured with: ../src/configure -v --with-pkgversion='Ubuntu 6.2.0-3ubuntu11~16.04' --with-bugurl=file:///usr/share/doc/gcc-6/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-6 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-6-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-6-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-6-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
    Thread model: posix
    gcc version 6.2.0 20160901 (Ubuntu 6.2.0-3ubuntu11~16.04) 
    
    
    
    duplicate 
    opened by abermea 23
  • module 'sonnet' has no attribute 'AbstractModule'

    module 'sonnet' has no attribute 'AbstractModule'

    Hi,I've installed sonnet by using pip install dm-sonnet However when I run "python evaluate_sample.py" I got AttributeError: module 'sonnet' has no attribute 'AbstractModule' Thanks!

    opened by poweryin 19
  • Hardcoded path in rnn_shakespear_test.py

    Hardcoded path in rnn_shakespear_test.py

    The path is here:

    https://github.com/deepmind/sonnet/blob/master/sonnet/examples/dataset_shakespeare.py#L88

    esc@omnitech ~/git/sonnet/sonnet/examples (git)-[master] % python rnn_shakespeare_test.py 
    E.
    ======================================================================
    ERROR: testRun (__main__.TinyShakespeareTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "rnn_shakespeare_test.py", line 31, in testRun
        rnn_shakespeare.train(10, 10, 9)
      File "/home/esc/git/sonnet/.venv/local/lib/python2.7/site-packages/sonnet/examples/rnn_shakespeare.py", line 180, in train
        name="shake_train")
      File "/home/esc/git/sonnet/.venv/local/lib/python2.7/site-packages/sonnet/examples/dataset_shakespeare.py", line 127, in __init__
        vocab_data_file=self._vocab_file)
      File "/home/esc/git/sonnet/.venv/local/lib/python2.7/site-packages/sonnet/examples/dataset_shakespeare.py", line 58, in __init__
        token_list = reading_function(vocab_data_file)
      File "/home/esc/git/sonnet/.venv/local/lib/python2.7/site-packages/sonnet/examples/dataset_shakespeare.py", line 53, in reading_function
        return list(f.read().replace("\n", self.CHAR_EOS))
      File "/home/esc/git/sonnet/.venv/local/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py", line 115, in read
        self._preread_check()
      File "/home/esc/git/sonnet/.venv/local/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py", line 75, in _preread_check
        compat.as_bytes(self.__name), 1024 * 512, status)
      File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
        self.gen.next()
      File "/home/esc/git/sonnet/.venv/local/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
        pywrap_tensorflow.TF_GetCode(status))
    NotFoundError: /cns/wj-d/home/deepmind/sequence_data/ts/ts.train.txt
    
    ----------------------------------------------------------------------
    Ran 2 tests in 0.004s
    
    FAILED (errors=1)
    
    opened by esc 15
  • Python3 compatibility?

    Python3 compatibility?

    Even if not officiously supported the codebase seems more or less python3 compatible already? Changing one line is enough to import sonnet in python3 and the Shakespeare example works fine after changing xrange.

    opened by bfredl 15
  • Fedora 24 unable to import sonnet

    Fedora 24 unable to import sonnet

    Fedora 24, tensorflow 1.0.1, bazel 0.4.5 Installing sonnet seems to have been successful (Requirement already satisfied: sonnet==1.0.....), including installing jdk8, bazel, sonnet, and the ./configure for tensorflow. But when I try to import sonnet, I get the error below. Any suggestions?

    import sonnet as snt Traceback (most recent call last): File "", line 1, in File "/usr/lib/python2.7/site-packages/sonnet/init.py", line 102, in from sonnet.python.ops.resampler import resampler File "/usr/lib/python2.7/site-packages/sonnet/python/ops/resampler.py", line 33, in tf.resource_loader.get_path_to_datafile("_resampler.so")) File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 64, in load_op_library None, None, error_msg, error_code) tensorflow.python.framework.errors_impl.NotFoundError: /usr/lib/python2.7/site-packages/sonnet/python/ops/_resampler.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringB5cxx11Ev

    However, works if I switch to the sonnet directory, then import, but then testing it I get an ImportError:

    import sonnet as snt import tensorflow as tf snt.resampler(tf.constant([0.]), tf.constant([0.])) Traceback (most recent call last): File "", line 1, in File "sonnet/python/ops/resampler.py", line 65, in resampler raise ImportError("_gen_resampler could not be imported.") ImportError: _gen_resampler could not be imported.

    I did uninstall sonnet before installing the whl file.

    opened by gabrielleyr 12
  • Correct way to install Sonnet gpu version (>= 1.26) and tensorflow probability >= 0.5.0

    Correct way to install Sonnet gpu version (>= 1.26) and tensorflow probability >= 0.5.0

    Hello DM team. I'm running a project on gcloud and I need some clarification about installation and dependencies. In the beginning I built my model on a dev machine with only CPU and dm-sonnet 1.26 and everything went fine. Than few days ago I was ready to train it so I moved to a new vm instance with 2 tesla P100 , python 3.5 and tensorflow-gpu 1.12 pre-installed. I've tried to update Sonnet to the latest version (1.29) and obviously I wanted to install the gpu version. Seems like the latest gpu version of Sonnet still the 1.26 (am I wrong ?)

    My problem is here. The installation process by pip3 crashes cause sonnet requires tensorflow-probability-gpu >= 0.4 and I get the message that there is no version that satisfies this requirement. so I did some research and I discovered that from version 0.5 tensorflow-probability and tensorflow-probability-gpu were merged in the same package.

    So my first question is: what's the right procedure to install the latest Sonnet-gpu version?

    second question: as a workaround (just cause I don't wanna be stucked) I've installed tensorflow-probaility 0.5.0 (that is the official version compatible with tf 1.12) and than I installed dm-sonnet (CPU) using pip3.

    I ran a train cycle just to check the workflow, I was expecting some errors (cause I didn't installed the right sonnet gpu version) but unexpectedly everything went fine. The input pipeline is showing optimal performance during the training on 2 GPUs the only thig is that while I'm building the model I recive a lot of deprecation warnings from numpy (i got the 1.16.1 version installed) like the following one.

    /home/manuel_migliazza/.local/lib/python3.5/site-packages/numpy/lib/type_check.py:546: DeprecationWarning: np.asscalar(a) is deprecated since NumPy v1.16, use a.item() instead 'a.item() instead', DeprecationWarning, stacklevel=1)

    opened by JCMiles 11
  • Build model with new TF dataset API

    Build model with new TF dataset API

    I was trying to fine-tuning I3D network using the new TensorFlow dataset/iterator API and get rid of the feed_dict approach. It seems the network cannot build since it can't understand input tensor dimensions. Here's some excerpt:

    [...]
    iterator = dataset.make_initializable_iterator()
    next_input, next_label, _, _ = iterator.get_next()
    with tf.variable_scope('RGB'):
      rgb_model = InceptionI3d(num_classes=10, spatial_squeeze=True, final_endpoint='Logits')
    logits, _ = rgb_model(rgb_input, is_training=False, dropout_keep_prob=FLAGS.dropout_keep_prob)
    [...]
    

    this produces the following error:

    File "/usr/local/lib/python3.5/dist-packages/sonnet/python/modules/conv.py", line 2714, in _build
        "Number of input channels must be known at module build time")
    sonnet.python.modules.base_errors.UnderspecifiedError: Number of input channels must be known at module build time
    

    However, the tensor next_input is correctly shaped in the form of [batch_size, frames, 224, 224, 3]. Can you see any error in my approach? Thank you

    opened by elmuz 11
  • _resampler.so: undefined symbol

    _resampler.so: undefined symbol

    faced this error while executing the example code

    import sonnet as snt
    import tensorflow as tf
    snt.resampler(tf.constant([0.]), tf.constant([0.]))
    

    recompiling the tensorflow to 1.1.0-rc2 and replacing the downloaded sonnet/tensorflow with this one solved the problem :)

    opened by animesh 11
  • Restoring w/ Adam?

    Restoring w/ Adam?

    I'm using tf.train.Saver() to save and restore a model. This works, except when training resumes, the accuracy of the model at first continues as it left off, but then suddenly drops completely as if starting from scratch. I checked everything in my code and I don't know what else could be responsible for it. I am using Adam and I wonder if those variables are not being restored because of some nuance of Sonnet. I can't figure out why training accuracy drops after a few batches with the restored model.

    opened by slerman12 9
  • cannot make sonnet working!!

    cannot make sonnet working!!

    Hi, I'm trying to simply install sonnet, but it is not working. I used instructions on the first page and also did sudo python setup.py build && python setup.py install

    it seems to be installed fine but I get the following error, can someone tell me what is going on?: `

    import sonnet as snt import tensorflow as tf snt.resampler(tf.constant([0.]), tf.constant([0.])) Traceback (most recent call last): File "", line 1, in File "sonnet/python/ops/resampler.py", line 65, in resampler raise ImportError("_gen_resampler could not be imported.") ImportError: _gen_resampler could not be imported.

    `

    opened by arnaghizadeh 9
  • Indicating training/testing modes in sonnet callbacks

    Indicating training/testing modes in sonnet callbacks

    What is a convenient way of providing boolean training flags, e.g. is_training that indicate, for example, whether to use batch_norm or not when using sonnet callback functions?

    Example:

    def make_transpose_cnn_model():
        def transpose_convnet1d(inputs):
            inputs = tf.expand_dims(inputs, axis=2)
    
            outputs = snt.Conv1DTranspose(output_channels=2, kernel_shape=10, stride=1)(inputs)
            outputs = snt.BatchNorm()(outputs, is_training=True) <- want to have this as input
            outputs = tf.nn.relu(outputs)
            outputs = snt.Conv1DTranspose(output_channels=2, kernel_shape=10, stride=1)(outputs)
            outputs = snt.BatchNorm()(outputs, is_training=True) <- want to have this as input
            outputs = tf.nn.relu(outputs)
            outputs = snt.BatchFlatten()(outputs)
            #outputs = tf.nn.dropout(outputs, keep_prob=tf.constant(1.0)) <- want to have this as input
            outputs = snt.Linear(output_size=128)(outputs)
    
            return outputs
    
        return transpose_convnet1d`
    

    and

    self._network = modules.GraphIndependent(
                    edge_model_fn=EncodeProcessDecode.make_mlp_model,
                    node_model_fn=EncodeProcessDecode.make_transpose_cnn_model,
                    global_model_fn = EncodeProcessDecode.make_mlp_model)
    

    I can't pass this parameter in the _build() function as shown in the following since the interface of modules.GraphIndipendent won't allow it:

        def _build(self, input_op, num_processing_steps, is_training=True):
            latent = self._encoder(input_op, is_training)
    
    

    it yields:

    TypeError: _build() got an unexpected keyword argument 'is_training'

    opened by ferreirafabio 8
  • Bump tensorflow from 2.5.1 to 2.9.3

    Bump tensorflow from 2.5.1 to 2.9.3

    Bumps tensorflow from 2.5.1 to 2.9.3.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.9.3

    Release 2.9.3

    This release introduces several vulnerability fixes:

    TensorFlow 2.9.2

    Release 2.9.2

    This releases introduces several vulnerability fixes:

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.9.3

    This release introduces several vulnerability fixes:

    Release 2.8.4

    This release introduces several vulnerability fixes:

    ... (truncated)

    Commits
    • a5ed5f3 Merge pull request #58584 from tensorflow/vinila21-patch-2
    • 258f9a1 Update py_func.cc
    • cd27cfb Merge pull request #58580 from tensorflow-jenkins/version-numbers-2.9.3-24474
    • 3e75385 Update version numbers to 2.9.3
    • bc72c39 Merge pull request #58482 from tensorflow-jenkins/relnotes-2.9.3-25695
    • 3506c90 Update RELEASE.md
    • 8dcb48e Update RELEASE.md
    • 4f34ec8 Merge pull request #58576 from pak-laura/c2.99f03a9d3bafe902c1e6beb105b2f2417...
    • 6fc67e4 Replace CHECK with returning an InternalError on failing to create python tuple
    • 5dbe90a Merge pull request #58570 from tensorflow/r2.9-7b174a0f2e4
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • I have a question about the perplexity term (in the VQ-VAE).

    I have a question about the perplexity term (in the VQ-VAE).

    As far as I understood, the perplexity used in this repo's VQ-VAE is kind of "meaningfully used codebook token numbers".

    When only one codebook token is used, perplexity is 1. When all codebook tokens appear uniformly, the perplexity equals the codebook nums.

    So I was wondering, for good output quality, what is the minimum threshold of "perplexity divided by codebook nums"? (I guess this should be found experimentally. If you have any results related to this question, it would be great to know.)

    opened by SeongYeonPark 2
  • (Maybe) in-consistency between VQ-VAE paper and its implementation.

    (Maybe) in-consistency between VQ-VAE paper and its implementation.

    FIrst of all, maybe it is my misunderstanding of the paper, so hope somebody could explain it for me, thanks! :


    in the paper, the loss is defined as Screenshot from 2022-08-30 11-52-26

    where e is the codebook defined at the beginning of the Section: Screenshot from 2022-08-30 11-57-36

    So, in the paper, the codebook loss and commitment loss are MSE between z_e(x) and e.

    However, in the implementation, they are implemented as MSE between z_e(x)(inputs) and z_q(x)(quantized), where variable quantized means quantized encoding of the image, namely z_q: Screenshot from 2022-08-30 11-58-19

    Are they actually the same thing? why?

    • If the paper stated is right. how the dimension matches between z_e(x)(H' * W' * D) and e(K * D)?
    • if the implementation is right. how z_q(x)(quantized) backprop since its calculation contains argmin?
    opened by Apollo1840 2
  • Can you explain the logic of updated_ema_cluster_size in VectorQuantizerEMA?

    Can you explain the logic of updated_ema_cluster_size in VectorQuantizerEMA?

    Hi, It seems updated_ema_cluster_size in VectorQuantizerEMA use small epsilon to avoid zero in denominator, but the formula is strange to me, so can you explain the logic here, and why not directly use something like x=x+epsilon? https://github.com/deepmind/sonnet/blob/d1cd37117bcb98223b3e4b930717d418abb76484/sonnet/src/nets/vqvae.py#L270-L271

    opened by wztdream 0
  • Cannot install sonnet on mac m1

    Cannot install sonnet on mac m1

    I install dm-sonnet and I see: Requirement already satisfied: sonnet in

    /Users/maryamkia/Homebrew/Caskroom/miniforge/base/envs/pytorch_x86/lib/python3.8/site-packages (0.1.6)
    Requirement already satisfied: networkx==1.8.1 in /Users/maryamkia/Homebrew/Caskroom/miniforge/base/envs/pytorch_x86/lib/python3.8/site-packages (from sonnet) (1.8.1)
    

    Note: you may need to restart the kernel to use updated packages.

    after reseting kernel and running:

    import sonnet as snt
    print("Sonnet version {}".format(snt.__version__))
    
    

    I got:

    AttributeError: module 'tensorflow' has no attribute 'GraphKeys' I work on mac M1

    opened by maryamkiashemshaki 3
  • this portion of attention code looks incorrect

    this portion of attention code looks incorrect

    attention_mlp = basic.BatchApply( mlp.MLP([self._mem_size] * self._attention_mlp_layers))

    for _ in range(self._num_blocks): attended_memory = self._multihead_attention(memory)

    shouldnt it be this

    attended_memory = memory for _ in range(self._num_blocks): attended_memory = self._multihead_attention(attended_memory)

    i know memory isn't changed in that function too, so isn't this expected to be redundant.

    opened by ava6969 0
Releases(v2.0.0)
  • v2.0.0(Mar 27, 2020)

    Sonnet 2 is a re-write of Sonnet for TensorFlow 2.0, it is built on top of tf.Module (tensorflow/community#56) enabling a simple and researcher friendly interface to TensorFlow.

    Source code(tar.gz)
    Source code(zip)
  • v2.0-beta(Sep 6, 2019)

    Sonnet 2 is a re-write of Sonnet for TensorFlow 2.0, it is built on top of tf.Module (tensorflow/community#56) enabling a simple and researcher friendly interface to TensorFlow.

    Closes #117 #123.

    Source code(tar.gz)
    Source code(zip)
This is a model made out of Neural Network specifically a Convolutional Neural Network model

This is a model made out of Neural Network specifically a Convolutional Neural Network model. This was done with a pre-built dataset from the tensorflow and keras packages. There are other alternative libraries that can be used for this purpose, one of which is the PyTorch library.

null 9 Oct 18, 2022
NeuPy is a Tensorflow based python library for prototyping and building neural networks

NeuPy v0.8.2 NeuPy is a python library for prototyping and building neural networks. NeuPy uses Tensorflow as a computational backend for deep learnin

Yurii Shevchuk 729 Jan 3, 2023
TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

TensorFlow Ranking is a library for Learning-to-Rank (LTR) techniques on the TensorFlow platform

null 2.6k Jan 4, 2023
This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

The Neural Process Family This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CN

DeepMind 892 Dec 28, 2022
Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayesian-Torch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.

Intel Labs 210 Jan 4, 2023
Neural-net-from-scratch - A simple Neural Network from scratch in Python using the Pymathrix library

A Simple Neural Network from scratch A Simple Neural Network from scratch in Pyt

Youssef Chafiqui 2 Jan 7, 2022
TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)

SLM: Structural Language Models of Code This is an official implementation of the model described in: "Structural Language Models of Code" [PDF] To ap

null 73 Nov 6, 2022
Use tensorflow to implement a Deep Neural Network for real time lane detection

LaneNet-Lane-Detection Use tensorflow to implement a Deep Neural Network for real time lane detection mainly based on the IEEE IV conference paper "To

MaybeShewill-CV 1.9k Jan 8, 2023
Tensorflow Implementation for "Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition"

Tensorflow Implementation for "Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition" Pre-trained Deep Convo

Ankush Malaker 5 Nov 11, 2022
A lightweight Python-based 3D network multi-agent simulator. Uses a cell-based congestion model. Calculates risk, loudness and battery capacities of the agents. Suitable for 3D network optimization tasks.

AMAZ3DSim AMAZ3DSim is a lightweight python-based 3D network multi-agent simulator. It uses a cell-based congestion model. It calculates risk, battery

Daniel Hirsch 13 Nov 4, 2022
JAX-based neural network library

Haiku: Sonnet for JAX Overview | Why Haiku? | Quickstart | Installation | Examples | User manual | Documentation | Citing Haiku What is Haiku? Haiku i

DeepMind 2.3k Jan 4, 2023
This is an open-source toolkit for Heterogeneous Graph Neural Network(OpenHGNN) based on DGL [Deep Graph Library] and PyTorch.

This is an open-source toolkit for Heterogeneous Graph Neural Network(OpenHGNN) based on DGL [Deep Graph Library] and PyTorch.

BUPT GAMMA Lab 519 Jan 2, 2023
Cascaded Pyramid Network (CPN) based on Keras (Tensorflow backend)

ML2 Takehome Project Reimplementing the paper: Cascaded Pyramid Network for Multi-Person Pose Estimation Dataset The model uses the COCO dataset which

Vo Van Tu 1 Nov 22, 2021
Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy.

Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy. Now with tensorflow 1.0 support. Evaluation usa

Marcel R. 349 Aug 6, 2022
Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Peter Lin 6.5k Jan 4, 2023
Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting (RVM) English | 中文 Official repository for the paper Robust High-Resolution Video Matting with Temporal Guidance. RVM is specific

flow-dev 2 Aug 21, 2022
A Tensorflow based library for Time Series Modelling with Gaussian Processes

Markovflow Documentation | Tutorials | API reference | Slack What does Markovflow do? Markovflow is a Python library for time-series analysis via prob

Secondmind Labs 24 Dec 12, 2022
Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

null 730 Jan 9, 2023