Code Repository for Liquid Time-Constant Networks (LTCs)

Last update: Dec 27, 2022

Overview

Liquid time-constant Networks (LTCs)

[Update] A Pytorch version is added in our sister repository: https://github.com/mlech26l/keras-ncp

This is the official repository for LTC networks described in paper: https://arxiv.org/abs/2006.04439 This repository alows you to train continuous-time models with backpropagation through-time (BPTT). Available Continuous-time models are:

Models	References
Liquid time-constant Networks	https://arxiv.org/abs/2006.04439
Neural ODEs	https://papers.nips.cc/paper/7892-neural-ordinary-differential-equations.pdf
Continuous-time RNNs	https://www.sciencedirect.com/science/article/abs/pii/S089360800580125X
Continuous-time Gated Recurrent Units (GRU)	https://arxiv.org/abs/1710.04110

Requisites

All models were implemented tested with TensorFlow 1.14.0 and python3 on Ubuntu 16.04 and 18.04 machines. All following steps assume that they are executed under these conditions.

Preparation

First we have to download all datasets by running

source download_datasets.sh

This script creates a folder data, where all downloaded datasets are stored.

Training and evaluating the models

There is exactly one python module per dataset:

Hand gesture segmentation: gesture.py
Room occupancy detection: occupancy.py
Human activity recognition: har.py
Traffic volume prediction: traffic.py
Ozone level forecasting: ozone.py

Each script accepts the following four agruments:

--model: lstm | ctrnn | ltc | ltc_rk | ltc_ex
--epochs: number of training epochs (default 200)
--size: number of hidden RNN units (default 32)
--log: interval of how often to evaluate validation metric (default 1)

Each script trains the specified model for the given number of epochs and evalutates the validation performance after every log steps. At the end of training, the best performing checkpoint is restored and the model is evaluated on the test set. All results are stored in the results folder by appending the result to CSV-file.

For example, we can train and evaluate the CT-RNN by executing

python3 har.py --model ctrnn

After the script is finished there should be a file results/har/ctrnn_32.csv created, containing the following columns:

best epoch: Epoch number that achieved the best validation metric
train loss: Training loss achieved at the best epoch
train accuracy: Training metric achieved at the best epoch
valid loss: Validation loss achieved at the best epoch
valid accuracy: Best validation metric achieved during training
test loss: Loss on the test set
test accuracy: Metric on the test set

Hyperparameters

Parameter	Value	Description
Minibatch size	16	Number of training samples over which the gradient descent update is computed
Learning rate	0.001/0.02	0.01-0.02 for LTC, 0.001 for all other models.
Hidden units	32	Number of hidden units of each model
Optimizer	Adam	See (Kingma and Ba, 2014)
beta_1	0.9	Parameter of the Adam method
beta_2	0.999	Parameter of the Adam method
epsilon	1e-08	Epsilon-hat parameter of the Adam method
Number of epochs	200	Maximum number of training epochs
BPTT length	32	Backpropagation through time length in time-steps
ODE solver sreps	1/6	relative to input sampling period
Validation evaluation interval	1	Interval of training epochs when the metrics on the validation are evaluated

Trajectory Length Analysis

Run the main.m file to get trajectory length results for the desired setting tuneable in the code.

Comments

contrain_op error when not using lct
I am using the LTC network for my data and everything works fine. However, when I am trying to use LSTM, CTRNN or CTGRU, this error appears:

File "seq.py", line 248, in fit if(len(self.constrain_op) > 0): TypeError: object of type 'NoneType' has no len()

Do you have any idea why is this happening? Thank you in advance for your time.
opened by prodangp 3

Tensorflow error in import tf.nn.rnn_cell

Traceback (most recent call last):
  File "c:\...\experiments_with_ltcs\ctrnn_model.py", line 105, in <module>
    class NODE(tf.nn.rnn_cell.RNNCell):
AttributeError: module 'tensorflow._api.v2.nn' has no attribute 'rnn_cell'

and I've to add, it's not solvable by replacing tf.nn.rnn_cell by tf.compat.v1.nn.rnn_cell Any suggestions?

opened by michaelkhany 2

License

Hello, this is a really nice model you have there. I would like to use it in one of my personal projects for research. Is it possible to add a license for this repository so that I know if I can use it or not ? Thanks

opened by alexbrillant 1
Traffic.py code with LTC is being killed and not running

Traffic.py code is being killed while running on Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1056-azure x86_64) Azure VM with Python-3.6.9 and tensorflow-cpu-1.14. Other algorithms like NODE, CT-RNN, LSTM, CT-GRU are running properly. Any such issue you faced while simulating the code? Please help.

opened by sumit421992 1
System Time-constant

The paper proved that the system time-constant is stable. I would like to check the system time-constant value via each epoch in source code. Do you provide any code function to validate the system time-constant value?

opened by huyenng308 0

Code Repository for Liquid Time-Constant Networks (LTCs)

Related tags

Overview

Liquid time-constant Networks (LTCs)

Requisites

Preparation

Training and evaluating the models

Hyperparameters

Trajectory Length Analysis

Comments

contrain_op error when not using lct

Tensorflow error in import tf.nn.rnn_cell

License

Traffic.py code with LTC is being killed and not running

System Time-constant

Owner

Ramin Hasani

Automatically replace ONNX's RandomNormal node with Constant node.

Simple Linear 2nd ODE Solver GUI - A 2nd constant coefficient linear ODE solver with simple GUI using euler's method

A very simple tool to rewrite parameters such as attributes and constants for OPs in ONNX models. Simple Attribute and Constant Modifier for ONNX.

This repository contains the source code for the paper "DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networks",

This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

[CVPR 2022] Official code for the paper: "A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibration"

This is the unofficial code of Deep Dual-resolution Networks for Real-time and Accurate Semantic Segmentation of Road Scenes. which achieve state-of-the-art trade-off between accuracy and speed on cityscapes and camvid, without using inference acceleration and extra data

Code for the paper "TadGAN: Time Series Anomaly Detection Using Generative Adversarial Networks"

This application is the basic of automated online-class-joiner(for YıldızEdu) within the right time. Gets the ZOOM link by scheduled date and time.

Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.

TCNN Temporal convolutional neural network for real-time speech enhancement in the time domain

Real-Time-Student-Attendence-System - Real Time Student Attendence System

This project is a loose implementation of paper "Algorithmic Financial Trading with Deep Convolutional Neural Networks: Time Series to Image Conversion Approach"

Deep learning-based approach to discovering Granger causality networks in multivariate time series

Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation

STMTrack: Template-free Visual Tracking with Space-time Memory Networks

Library for implementing reservoir computing models (echo state networks) for multivariate time series classification and clustering.

An implementation of paper `Real-time Convolutional Neural Networks for Emotion and Gender Classification` with PaddlePaddle.

Pytorch implementation of "Forward Thinking: Building and Training Neural Networks One Layer at a Time"