Lattice methods in TensorFlow

Overview

TensorFlow Lattice

TensorFlow Lattice is a library that implements constrained and interpretable lattice based models. It is an implementation of Monotonic Calibrated Interpolated Look-Up Tables in TensorFlow.

The library enables you to inject domain knowledge into the learning process through common-sense or policy-driven shape constraints. This is done using a collection of Keras layers that can satisfy constraints such as monotonicity, convexity and pairwise trust:

  • PWLCalibration: piecewise linear calibration of signals.
  • CategoricalCalibration: mapping of categorical inputs into real values.
  • Lattice: interpolated look-up table implementation.
  • Linear: linear function with monotonicity and norm constraints.

The library also provides easy to setup canned estimators for common use cases:

  • Calibrated Linear
  • Calibrated Lattice
  • Random Tiny Lattices (RTL)
  • Crystals

With TF Lattice you can use domain knowledge to better extrapolate to the parts of the input space not covered by the training dataset. This helps avoid unexpected model behaviour when the serving distribution is different from the training distribution.

You can install our prebuilt pip package using

pip install tensorflow-lattice
Comments
  • Unable to execute example program

    Unable to execute example program

    I have installed tensorflow-lattice using pip 9.0.1 in Python 3.5.2 on Ubuntu 16.04 LTS. Tensorflow version is 1.3.1. For testing purpose I tried to execute example program

    import tensorflow as tf
    import tensorflow_lattice as tfl
    
    x = tf.placeholder(tf.float32, shape=(None, 2))
    (y, _, _, _) = tfl.lattice_layer(x, lattice_sizes=(2, 2))
    
    with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
      print(sess.run(y, feed_dict={x: [[0.0, 0.0]]}))
    

    which resulted in error. Here is stack trace from Jupyter notebook

    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-7-444d1bededea> in <module>()
    ----> 1 (y, _, _, _) = tfl.lattice_layer(x, lattice_sizes=(2, 2))
    
    /usr/local/lib/python3.5/dist-packages/tensorflow_lattice/python/lib/lattice_layers.py in lattice_layer(input_tensor, lattice_sizes, is_monotone, output_dim, interpolation_type, lattice_initializer, l1_reg, l2_reg, l1_torsion_reg, l2_torsion_reg, l1_laplacian_reg, l2_laplacian_reg)
        193   parameter_tensor = variable_scope.get_variable(
        194       interpolation_type + '_lattice_parameters',
    --> 195       initializer=lattice_initializer)
        196 
        197   output_tensor = lattice_ops.lattice(
    
    /usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py in get_variable(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)
       1063       collections=collections, caching_device=caching_device,
       1064       partitioner=partitioner, validate_shape=validate_shape,
    -> 1065       use_resource=use_resource, custom_getter=custom_getter)
       1066 get_variable_or_local_docstring = (
       1067     """%s
    
    /usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)
        960           collections=collections, caching_device=caching_device,
        961           partitioner=partitioner, validate_shape=validate_shape,
    --> 962           use_resource=use_resource, custom_getter=custom_getter)
        963 
        964   def _get_partitioned_variable(self,
    
    /usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)
        365           reuse=reuse, trainable=trainable, collections=collections,
        366           caching_device=caching_device, partitioner=partitioner,
    --> 367           validate_shape=validate_shape, use_resource=use_resource)
        368 
        369   def _get_partitioned_variable(
    
    /usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py in _true_getter(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource)
        350           trainable=trainable, collections=collections,
        351           caching_device=caching_device, validate_shape=validate_shape,
    --> 352           use_resource=use_resource)
        353 
        354     if custom_getter is not None:
    
    /usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variable_scope.py in _get_single_variable(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource)
        662                          " Did you mean to set reuse=True in VarScope? "
        663                          "Originally defined at:\n\n%s" % (
    --> 664                              name, "".join(traceback.format_list(tb))))
        665       found_var = self._vars[name]
        666       if not shape.is_compatible_with(found_var.get_shape()):
    
    ValueError: Variable hypercube_lattice_parameters already exists, disallowed. Did you mean to set reuse=True in VarScope? Originally defined at:
    
      File "/usr/local/lib/python3.5/dist-packages/tensorflow_lattice/python/lib/lattice_layers.py", line 195, in lattice_layer
        initializer=lattice_initializer)
      File "<ipython-input-1-e860b057ec64>", line 5, in <module>
        (y, _, _, _) = tfl.lattice_layer(x, lattice_sizes=(2, 2))
      File "/usr/local/lib/python3.5/dist-packages/IPython/core/interactiveshell.py", line 2847, in run_code
        exec(code_obj, self.user_global_ns, self.user_ns)
    
    
    opened by kamilmahmood 22
  • _pwl_calibration_ops.so image not found

    _pwl_calibration_ops.so image not found

    I just installed tensorflow-lattice on a MacOS but got the following importing error. Do you know what's happening here?

    Python 3.6.8 |Anaconda, Inc.| (default, Dec 29 2018, 19:04:46) [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin Type "help", "copyright", "credits" or "license" for more information.

    import tensorflow_lattice /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) /anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) Traceback (most recent call last): File "", line 1, in File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/init.py", line 33, in from tensorflow_lattice.python.estimators.calibrated import input_calibration_layer_from_hparams File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/estimators/calibrated.py", line 28, in from tensorflow_lattice.python.lib import pwl_calibration_layers File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/lib/pwl_calibration_layers.py", line 36, in from tensorflow_lattice.python.ops import pwl_calibration_ops File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/ops/pwl_calibration_ops.py", line 45, in '../../cc/ops/_pwl_calibration_ops.so')) File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/ops/../../cc/ops/_pwl_calibration_ops.so, 6): image not found

    opened by gyz0807-ai 10
  • Cannot save keras model with tensorflow lattice layers

    Cannot save keras model with tensorflow lattice layers

    Saving the model having keras tfl layers creates the following problem.

    `/usr/local/lib/python3.6/dist-packages/h5py/_hl/group.py in setitem(self, name, obj) 371 372 if isinstance(obj, HLObject): --> 373 h5o.link(obj.id, self.id, name, lcpl=lcpl, lapl=self._lapl) 374 375 elif isinstance(obj, SoftLink):

    h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

    h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

    h5py/h5o.pyx in h5py.h5o.link()

    RuntimeError: Unable to create link (name already exists)`

    The error is reproduced in the colab example here. https://colab.research.google.com/drive/1tknejj9CtM27bHGktsZSTnvLvH3eCgG8

    opened by devavratTomar 7
  • TensorFlow 2.0 plan

    TensorFlow 2.0 plan

    Is there any plan for tensorflow 2.0 release? I heard that the eager mode will be default in TF 2.0, but not sure TF lattice is ready for it.

    Thank you!

    opened by si-you 7
  • setup.py fix sklearn → scikit-learn

    setup.py fix sklearn → scikit-learn

    The package name is scikit-learn while the import is sklearn.

    See https://pypi.org/project/sklearn/ for the official recommendation.

    Requiring sklearn can lead to subtle problems as explained in https://github.com/scikit-learn/scikit-learn/issues/8215 .

    opened by maresb 5
  • Optimizer helper functions

    Optimizer helper functions

    SGD or Adagrad with a square root decay learning rate schedule can help training. Since TensorFlow lattice estimators accepts arbitrary callable as an optimizer, so we can use learning rate scheduling, but it is not easy to configure for a beginner. This pull requests include some helper functions and tests to illustrate how to use a custom learning rate schedule.

    Also this pull requests include modify keep_dims argument to keepdims in linear model construction since tensorflow is deprecating a former argument.

    opened by si-you 5
  • Issue with gast

    Issue with gast

    I installed tensorflow lattice with TF 2.3.0 but when trying to run I receive a conversion error and an attribute error "gast has no attribute 'Index'"

    I tried to install three versions of tensorflow lattice (0.9.9, 2.0, and 2.0.8) but recieved the same error and I could not install an older version of gast (currently running 0.4.0) due to the dependencies of my current setup.

    Please let me know if you have additional recommendations. Thank you!

    opened by josem789 4
  • Error in running the example of lattice models

    Error in running the example of lattice models

    I was running the uci_census.py file, with the create_calibrated_lattice function. When parameter lattice_size is set to 2, the program can run successfully. However, when the parameter is set to 3 (also 4 or other values, which I have not tested yet), the program will crash with the following error:

    2018-06-17 19:54:25.814852: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA
    Traceback (most recent call last):
      File "uci_census.py", line 616, in <module>
        run()
      File "uci_census.py", line 609, in run
        main(argv)
      File "uci_census.py", line 586, in main
        train(estimator)
      File "uci_census.py", line 550, in train
        batch_size=FLAGS.batch_size, num_epochs=epochs, shuffle=True))
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 314, in train
        loss = self._train_model(input_fn, hooks, saving_listeners)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 812, in _train_model
        log_step_count_steps=self._config.log_step_count_steps) as mon_sess:
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 380, in MonitoredTrainingSession
        stop_grace_period_secs=stop_grace_period_secs)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 787, in __init__
        stop_grace_period_secs=stop_grace_period_secs)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 511, in __init__
        self._sess = _RecoverableSession(self._coordinated_creator)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 972, in __init__
        _WrappedSession.__init__(self, self._create_session())
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 977, in _create_session
        return self._sess_creator.create_session()
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 668, in create_session
        self.tf_sess = self._session_creator.create_session()
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 440, in create_session
        init_fn=self._scaffold.init_fn)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/session_manager.py", line 273, in prepare_session
        config=config)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/session_manager.py", line 205, in _restore_checkpoint
        saver.restore(sess, ckpt.model_checkpoint_path)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1686, in restore
        {self.saver_def.filename_tensor_name: save_path})
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
        run_metadata_ptr)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1128, in _run
        feed_dict_tensor, options, run_metadata)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
        options, run_metadata)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
        raise type(e)(node_def, op, message)
    tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [1,1594323] rhs shape= [1,8192]
    	 [[Node: save/Assign_3 = Assign[T=DT_FLOAT, _class=["loc:@calibrated_tf_lattice_model/lattice/hypercube_lattice_parameters"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](calibrated_tf_lattice_model/lattice/calibrated_tf_lattice_model/lattice/hypercube_lattice_parameters/Adam_1, save/RestoreV2_3)]]
    
    Caused by op u'save/Assign_3', defined at:
      File "uci_census.py", line 616, in <module>
        run()
      File "uci_census.py", line 609, in run
        main(argv)
      File "uci_census.py", line 586, in main
        train(estimator)
      File "uci_census.py", line 550, in train
        batch_size=FLAGS.batch_size, num_epochs=epochs, shuffle=True))
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 314, in train
        loss = self._train_model(input_fn, hooks, saving_listeners)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 812, in _train_model
        log_step_count_steps=self._config.log_step_count_steps) as mon_sess:
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 380, in MonitoredTrainingSession
        stop_grace_period_secs=stop_grace_period_secs)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 787, in __init__
        stop_grace_period_secs=stop_grace_period_secs)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 511, in __init__
        self._sess = _RecoverableSession(self._coordinated_creator)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 972, in __init__
        _WrappedSession.__init__(self, self._create_session())
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 977, in _create_session
        return self._sess_creator.create_session()
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 668, in create_session
        self.tf_sess = self._session_creator.create_session()
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 431, in create_session
        self._scaffold.finalize()
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 212, in finalize
        self._saver.build()
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1248, in build
        self._build(self._filename, build_save=True, build_restore=True)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1284, in _build
        build_save=build_save, build_restore=build_restore)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 759, in _build_internal
        restore_sequentially, reshape)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 471, in _AddShardedRestoreOps
        name="restore_shard"))
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 440, in _AddRestoreOps
        assign_ops.append(saveable.restore(tensors, shapes))
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 160, in restore
        self.op.get_shape().is_fully_defined())
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 276, in assign
        validate_shape=validate_shape)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 59, in assign
        use_locking=use_locking, name=name)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
        op_def=op_def)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3160, in create_op
        op_def=op_def)
      File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1625, in __init__
        self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access
    
    InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1,1594323] rhs shape= [1,8192]
    	 [[Node: save/Assign_3 = Assign[T=DT_FLOAT, _class=["loc:@calibrated_tf_lattice_model/lattice/hypercube_lattice_parameters"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](calibrated_tf_lattice_model/lattice/calibrated_tf_lattice_model/lattice/hypercube_lattice_parameters/Adam_1, save/RestoreV2_3)]]
    

    IMO, the point should be this line: Assign requires shapes of both tensors to match. lhs shape= [1,1594323] rhs shape= [1,8192], in which 1594323 = 3^13 and 8192 = 2^13. Here 13 is the number of features used in this example, and 3 is the lattice_size we defined. Could anyone help me with this?

    opened by arrowx123 4
  • Using lattice in tf serving

    Using lattice in tf serving

    Currently when trying to serve a lattice model with tf serving, I run into an op that isn't supported in the serving kernel

    ... 2018-03-01 23:45:59.827196: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:284] Loading SavedModel: fail. Took 429225 microseconds. 2018-03-01 23:45:59.828892: E tensorflow_serving/util/retrier.cc:38] Loading servable: {name: default version: 1519947547} failed: Not found: Op type not registered 'PwlIndexingCalibrator' in binary running on dsexperiment-prod-0fe24ce9bf2552633. Make sure the Op and Kernel are registered in the binary running in this process.

    Both tf and tf-serving on the system are at version 1.5.0. Are lattice models not supported with serving yet ? If they are, could you point me to how to make it happen.

    opened by fabrol 4
  • convex by pieces function

    convex by pieces function

    Hi,

    I wonder if you have a functionality to specify that the target function should be convex by pieces, and/or monotonic by pieces.

    Thanks for writing this amazing piece of software :) Matias

    opened by matibilkis 3
  • How to use multi CPU easily?

    How to use multi CPU easily?

    It is so great to see such a good package. However the speed is too slow.

    I am using Crystal ensemble model config. tfl.estimators.CannedRegressor estimator. It seems only one CPU is using, though I have 48 CPUs on the machine.

    I have set the dataset with multiple threads:

    feature_analysis_input_fn = tf.compat.v1.estimator.inputs.pandas_input_fn(
        x=train_xs.loc[feature_analysis_index].copy(), 
        y=train_ys.loc[feature_analysis_index].copy(), 
        batch_size=128, 
        num_epochs=1, 
        shuffle=True, 
        queue_capacity=1000,
        num_threads=40)
    
    prefitting_input_fn = tf.compat.v1.estimator.inputs.pandas_input_fn(
        x=train_xs.loc[prefitting_index].copy(), 
        y=train_ys.loc[prefitting_index].copy(), 
        batch_size=128, 
        num_epochs=1, 
        shuffle=True, 
        queue_capacity=1000,
        num_threads=40)
    
    train_input_fn = tf.compat.v1.estimator.inputs.pandas_input_fn(
        x=train_xs.loc[train_index].copy(), 
        y=train_ys.loc[train_index].copy(), 
        batch_size=128, 
        num_epochs=100, 
        shuffle=True, 
        queue_capacity=1000,
        num_threads=40)
    

    The usage of CPU is still only 1.25 CPU. Any suggestion?

    opened by fuyanlu1989 3
  • experiment result explanation in tutorial

    experiment result explanation in tutorial

    Hi, I'm looking at the shape constraints tutorial, and the results for GBDT and DNN are listed in the tutorial as follows:

    • GBT Validation AUC: 0.7248634099960327
    • GBT Test AUC: 0.6980501413345337
    • DNN Validation AUC: 0.7518489956855774
    • DNN Testing AUC: 0.745200514793396

    After the experiment results, the tutorial comments Note that even though the validation metric is better than the tree solution, the testing metric is much worse.

    I don't understand where this comment comes from, since DNN outperforms GBT in both validation AUC and testing AUC.

    opened by liangchen1ceeee 2
  • Many-batches predictions

    Many-batches predictions

    Hi,

    When trying to get predictions of Lattice Models on more than one batch of data at once, Errors are raised. This is a nice feature to efficiently get predictions, and is present in basic Neural Network Keras models; find some examples in this colab.

    As far as I can tell from looking at API docs + source code, this should be related to the inputs admitted by PWC layers, but I wonder if there is an easy way around.

    In particular, this piece of code captures what I would like to get (and retrieves an error when calling on batched_inputs):

    
    class LatticeModel(tf.keras.Model):
        def __init__(self, nodes=[2,2], nkeypoints=100):
            super(LatticeModel,self).__init__()
            self.combined_calibrators = tfl.layers.ParallelCombination()
            for ind,i in enumerate(range(2)):
              calibration_layer = tfl.layers.PWLCalibration(input_keypoints=np.linspace(0,1,nkeypoints),output_min=0.0, output_max=nodes[ind])
              self.combined_calibrators.append(calibration_layer)
            self.lattice = tfl.layers.Lattice(lattice_sizes=nodes,interpolation="simplex")
            
        def call(self, x):
            rescaled = self.combined_calibrators(x)
            feat = self.lattice(rescaled)
            return feat
        
    #we define some input data
    x1 = np.random.randn(100,1).astype(np.float32)
    x2 = np.random.randn(100,1).astype(np.float32)
    
    inputs = tf.concat([x1,x2], axis=-1)
    
    #we initialize out model, and feed it with a batch of size 100
    model = LatticeModel()
    model(inputs)
    
    ### now we would like to efficiently predict the output of the lattice model on many batches of data at once (in this case 2)
    batched_inputs = np.random.randn(2,100,1)
    model(batched_inputs)
    

    Thanks a lot! Matías.

    opened by matibilkis 2
  • [*.py] Rename

    [*.py] Rename "Arguments:" to "Args:"

    I've written custom parsers and emitters for everything from docstrings to classes and functions. However, I recently came across an issue with the TensorFlow codebase: inconsistent use of Args: and Arguments: in its docstrings. It is easy enough to extend my parsers to support both variants, however it looks like Arguments: is wrong anyway, as per:

    • https://google.github.io/styleguide/pyguide.html#doc-function-args @ ddccc0f

    • https://chromium.googlesource.com/chromiumos/docs/+/master/styleguide/python.md#describing-arguments-in-docstrings @ 9fc0fc0

    • https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html @ c0ae8e3

    Therefore, only Args: is valid. This PR replaces them throughout the codebase.

    PS: For related PRs, see tensorflow/tensorflow/pull/45420

    opened by SamuelMarks 0
  • Feature Request - Is there a way to enforce an S-shape constraint ?

    Feature Request - Is there a way to enforce an S-shape constraint ?

    First off - Thank you so much for open sourcing Tensorflow lattice! It is great to make use of lattice interpolation to enforce predicate domain knowledge concerning monotonicity and convexity. Looking through the current documentation, I see it is possible to enforce an increasing and concave graph for diminishing returns, but what if I want to enforce an S-curve (i.e. an increasing convex curve with an inflection point that then turns concave)?

    opened by marwan116 3
  • Link to API docs 404s

    Link to API docs 404s

    https://github.com/tensorflow/lattice/blob/master/docs/overview.md#tutorials-and-api-docs points to https://github.com/tensorflow/lattice/blob/master/docs/api_docs/python/tfl.ipynb

    opened by kevinykuo 1
Releases(v2.0.11)
  • v2.0.11(Oct 20, 2022)

    Changes:

    • Updating code, tests and tutorials to support changes to tf.keras.optimizers.
    • Documentation updates.
    • Minor bug fixes.

    PyPI Release:

    • Generic package for py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.10(Jan 13, 2022)

    Changes:

    • Support for weighted quantiles for Estimators and Premade.
    • Helper functions for computing quantiles in premade_lib
    • Documentation updates.
    • Minor bug fixes.

    PyPI Release:

    • Generic package for py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.9(Sep 30, 2021)

    Changes:

    • (experimental) Cumulative Distribution Function (CDF) Layer that supports projection free monotonicity.
    • 'input_keypoints_type' parameter for PWLCalibration integration with Premade/Estimator models.
    • Estimator support for tf.data.Dataset inputs.
    • General tutorial/code cleanup.
    • Typo fixes.
    • Bug fixes.

    PyPI Release:

    • Generic package for py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.8(Feb 17, 2021)

    Changes:

    • (experimental) Parameterization option for Premade/Estimators that enables the use of both normal tfl.layers.Lattice layers ('all_vertices') and tfl.layers.KroneckerFactoredLattice layers ('kronecker_factored').
    • (experimental) KroneckerFactoredLattice layer visualization support for Estimators.
    • (experimental) KroneckerFactoredLattice bound constraints.
    • 'input_keypoints_type' parameter for PWLCalibration layers that enables learned input keypoints ('learned_interior') or the original fixed keypoints ('fixed').
    • General tutorial/code cleanup
    • Typo fixes
    • Bug fixes

    PyPI Release:

    • Generic package for py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.7(Dec 14, 2020)

    Changes:

    • (experimental) KroneckerFactoredLattice initialization now sorts on kernel axis 1 such that we sort each term individually.
    • (experimental) KroneckerFactoredLattice initialization defaults to [0.5, 1.5] instead of [0,1].
    • (experimental) KroneckerFactoredLattice custom_reduce_prod in interpolation for faster gradient computations.
    • Update bound and trust projection algorithms to compute violations for each unit separately.
    • 'loss_fn' option for estimators to use custom loss without having to define a custom head.
    • Enable calibrators to return a list of outputs per unit.
    • Enable RTL layer to return non-averaged outputs.
    • General tutorial/code cleanup
    • Typo fixes
    • Bug fixes

    PyPI Release:

    • Generic package for py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.6(Aug 10, 2020)

    TensorFlow is dropping py2 support, so we will be dropping support as well in our future releases. This is the last release that will support py2.

    Changes:

    • New (experimental) KroneckerFactoredLattice Layer, which introduces a new parameterization of our Lattice layer with linear space/time complexity.
    • rtl_lib.py helper functions for RTL Layer.
    • Utils module with useful helper functions for all layers.
    • 'rtl_layer' option for CalibratedLatticeEnsemble Premade Models and Canned Estimators, which uses an RTL Layer for the underlying ensemble. Can potentially give a speed-boost for models with a large number of lattices.
    • General code cleanup
    • Typo fixes
    • Bug fixes

    PyPI release:

    • Generic package for py2/py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.5(Jun 15, 2020)

    Changes:

    • Simplex interpolation support for lattices: O(d log(d)) simplex interpolation compared to O(2^d) hypercube interpolation is 2-10x faster with similar or improved training loss.
    • RTL layer performance optimization: 2-3x faster and scales much better with wider and deeper models with tens of thousands of lattices.
    • Optimization of 2^D hypercube lattices: 10-15% speedup.
    • PWL Calibration Sonnet Module (more to come in follow up releases)
    • New aggregation function tutorial
    • Linear combination support for canned ensemble models.
    • Improvement and bug fixes for save/load functionality
    • Bug fixes

    PyPI release:

    • Generic package for py2/py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.4(Apr 14, 2020)

    Changes:

    • Save/load support for Keras models (HDF5/H5 format)
    • RTL layer: An ensemble of Lattice layers that takes in a collection of monotonic and unconstrained features and randomly arranges them into lattices of a given rank.
    • AggregateFunction Premade model and Aggregation layer: Applies monotonic function on set inputs passed in as ragged tensors.
    • Crystals Lattice ensemble with Premade model
    • Feature updates to Lattice layer
    • Updates to tutorials
    • Bug fixes

    PyPI release:

    • Generic package for py2/py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.3(Mar 6, 2020)

    Changes:

    • Two new tutorials: premade models, shape constraints for ML fairness
    • Improvements and additions to premade models
    • New range dominance for Lattice and Linear layers
    • Added 'peak' mode to unimodality constraint
    • Updates to documentation
    • Bug fixes

    PyPI release:

    • Generic package for py2/py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.2(Feb 8, 2020)

    Changes:

    • Adding premade Keras models in tfl.premade module.
    • Adding RandomMonotonicInitializer for lattices.
    • Several edits to tutorials and API docs.

    PyPI release:

    • Generic package for py2/py3 that should work for TF 1.15 or TF 2.x.

    Notes:

    • The API for the premade Keras Models is experimental.
    • Creating premade models currently require a fully specified model configuration. We plan to use the new preprocessing mechanism in Keras to support keypoint initialization in future releases.
    Source code(tar.gz)
    Source code(zip)
  • v2.0.1(Feb 4, 2020)

    Changes:

    • Several edits to tutorials and API docs.
    • Bug fixes.

    PyPI release:

    • Generic package for py2/py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v2.0(Jan 28, 2020)

    This is a completely new implementation of the TensorFlow Lattice library. It is not backwards compatible with the previous versions of the library.

    Changes:

    • Core TF implementation: TFL v2 is a python-only library with all operations implemented in core TensorFlow, making it compatible with any platform that can run TensorFlow (cpu, gpu, tpu).
    • Keras layers: The new library provides Keras layers that can be mixed and matched with other Keras layers and used in Keras models. All constraints and regularizers are handled through Keras mechanisms and should work seamlessly without the need for hooks or callbacks.
    • New and improved canned estimators: The new library has a new simplified API for creating canned estimators. Calculation of feature quantiles is now automated in estimator construction. A version of the Crystals algorithm is now supported.
    • New constraint types: Several new types of constraints are added to the library, including convexity, unimodality, and pairwise feature trust and dominance relations.
    • Improved documentation and tutorials: Examples and tutorials are provided as notebooks. All documentations, examples, and API docs will be available on tensorflow.org.
    • Faster release cycle: With the library implemented in core TF, we hope to be able to release updates and improvements more frequently.

    Notes:

    • Some of the new 2-dimensional constraints are under active development and might undergo API changes.

    PyPI release:

    • Generic package for py2/py3 that should work for TF 1.15 or TF 2.x.
    Source code(tar.gz)
    Source code(zip)
  • v0.9.9(Jul 31, 2019)

    Changes:

    • Updating the code base for TF 2.0 compatibility in tf.compat.v1 mode
    • Changing tensorflow branch to 1.14
    • Changes to build scripts for bazel 0.25.2
    • Bug fixes

    PyPI release:

    • Includes python 2.7 and python 3 on macos and ubuntu
    • No gpu binary package is released with 0.9.9

    Important Note:

    This is the last release of the current version of the Tensorflow Lattice library. A new version of the library will be released soon:

    • Eager compatible base lattice and calibration library implemented in core TF (no custom ops)
    • Includes Keras layer and canned estimators
    • Not backwards compatible with the current version, but conversion should be easy
    Source code(tar.gz)
    Source code(zip)
  • v0.9.8(Oct 8, 2018)

  • v0.9.7(Jul 30, 2018)

  • v0.9.6(Feb 15, 2018)

    • New Estimators for separately-calibrated random tiny lattices: each lattices has its own calibrators for each input feature.
    • Updating TensorFlow submodule to r1.5.
    • Bug fixes.
    Source code(tar.gz)
    Source code(zip)
  • v0.9.5(Feb 1, 2018)

  • v0.9.4(Nov 9, 2017)

    • Update BUILD rules to work with TensorFlow 1.4.
    • Compile the binary package targeting TensorFlow 1.4 branch.
    • Changed the bias term in lattice initialization strategy from -0.5 to 0.0.
    Source code(tar.gz)
    Source code(zip)
  • v0.9.3(Oct 18, 2017)

Owner
null
A number of methods in order to perform Natural Language Processing on live data derived from Twitter

A number of methods in order to perform Natural Language Processing on live data derived from Twitter

null 1 Nov 24, 2021
Transformer-based Text Auto-encoder (T-TA) using TensorFlow 2.

T-TA (Transformer-based Text Auto-encoder) This repository contains codes for Transformer-based Text Auto-encoder (T-TA, paper: Fast and Accurate Deep

Jeong Ukjae 13 Dec 13, 2022
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 77.3k Jan 3, 2023
Making text a first-class citizen in TensorFlow.

TensorFlow Text - Text processing in Tensorflow IMPORTANT: When installing TF Text with pip install, please note the version of TensorFlow you are run

null 1k Dec 26, 2022
Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

Texar is a toolkit aiming to support a broad set of machine learning, especially natural language processing and text generation tasks. Texar provides

ASYML 2.3k Jan 7, 2023
🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

State-of-the-art Natural Language Processing for PyTorch and TensorFlow 2.0 ?? Transformers provides thousands of pretrained models to perform tasks o

Hugging Face 40.9k Feb 18, 2021
Making text a first-class citizen in TensorFlow.

TensorFlow Text - Text processing in Tensorflow IMPORTANT: When installing TF Text with pip install, please note the version of TensorFlow you are run

null 692 Feb 16, 2021
Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/

Texar is a toolkit aiming to support a broad set of machine learning, especially natural language processing and text generation tasks. Texar provides

ASYML 2.1k Feb 17, 2021
A collection of Korean Text Datasets ready to use using Tensorflow-Datasets.

tfds-korean A collection of Korean Text Datasets ready to use using Tensorflow-Datasets. TensorFlow-Datasets를 이용한 한국어/한글 데이터셋 모음입니다. Dataset Catalog |

Jeong Ukjae 20 Jul 11, 2022
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

English | 简体中文 | 繁體中文 State-of-the-art Natural Language Processing for Jax, PyTorch and TensorFlow ?? Transformers provides thousands of pretrained mo

Hugging Face 77.2k Jan 3, 2023
Implementing SimCSE(paper, official repository) using TensorFlow 2 and KR-BERT.

KR-BERT-SimCSE Implementing SimCSE(paper, official repository) using TensorFlow 2 and KR-BERT. Training Unsupervised python train_unsupervised.py --mi

Jeong Ukjae 27 Dec 12, 2022
GSoC'2021 | TensorFlow implementation of Wav2Vec2

GSoC'2021 | TensorFlow implementation of Wav2Vec2

Vasudev Gupta 73 Nov 28, 2022
Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

null 730 Jan 9, 2023
State of the art faster Natural Language Processing in Tensorflow 2.0 .

tf-transformers: faster and easier state-of-the-art NLP in TensorFlow 2.0 ****************************************************************************

null 74 Dec 5, 2022
A list of NLP(Natural Language Processing) tutorials built on Tensorflow 2.0.

A list of NLP(Natural Language Processing) tutorials built on Tensorflow 2.0.

Won Joon Yoo 335 Jan 4, 2023
Tensorflow Implementation of A Generative Flow for Text-to-Speech via Monotonic Alignment Search

Tensorflow Implementation of A Generative Flow for Text-to-Speech via Monotonic Alignment Search

Ankur Dhuriya 10 Oct 13, 2022
Python script to generate vector graphics of an oriented lattice unit cell

unitcell Python script to generate vector graphics of an oriented lattice unit cell Examples unitcell --type hexagonal --eulers 12 23 34 --axes --crys

Philip Eisenlohr 2 Dec 10, 2021
aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)

Bayesian Methods for Hackers Using Python and PyMC The Bayesian method is the natural approach to inference, yet it is hidden from readers behind chap

Cameron Davidson-Pilon 25.1k Jan 2, 2023
A PyTorch-based open-source framework that provides methods for improving the weakly annotated data and allows researchers to efficiently develop and compare their own methods.

Knodle (Knowledge-supervised Deep Learning Framework) - a new framework for weak supervision with neural networks. It provides a modularization for se

null 93 Nov 6, 2022
Here, I have discuss the three methods of list reversion. The three methods are built-in method, slicing method and position changing method.

Three-different-method-for-list-reversion Here, I have discuss the three methods of list reversion. The three methods are built-in method, slicing met

Sachin Vinayak Dabhade 4 Sep 24, 2021