duralava is a neural network which can simulate a lava lamp in an infinite loop.

Maximilian Bachl

Last update: Dec 20, 2022

Related tags

Deep Learning duralava

Overview

duralava

duralava is a neural network which can simulate a lava lamp in an infinite loop.

Example

This is not a real lava lamp but a "fake" one generated by duralava.

Novelty

duralava can

learn a physical process (a lava lamp).
generate an arbitarily long sequence of output, without diverging even after hours (outputting tens of thousands of frames).

How it works

Generative Adversarial Networks (GANs) can learn to generate new samples of data. For example, a GAN can be trained to output images of a lava lamp which look as real as possible. To accomplish this, the GAN gets an input vector with normally distributed noise. For duralava this vector is of length 64. Based on this random noise vector it generates a lava lamp image. The random vector thus encodes the state of the lava lamp.

For training, the GAN is presented a real image of a lava lamp and also one of the fake lava lamp and then it learns to make the fake ones look as real as possible.

For a lava lamp, a sequence of images has to be created. This sequence should in fact be infinite since a lava lamp can run forever. Thus the GAN should learn to output an arbitrarily long sequence of lava lamp images as a video. This is achieved by using a recurrent neural network (RNN). The RNN gets the 64 element noise vector of time step t and outputs the 64 element noise vector for time stemp t+1.

The tricky part is to make sure that the state of the lava lamp (the 64 element random noise vector) remains stable. It could for example happen that over time the distribution of noise in the vector diverges from a normal distribution the mean becomes 10 and the standard deviation 52. In this case, the output images of the lava lamps wouldn't be correct anymore as the GAN was trained to expect the input vector to be normally distributed. To solve this problem, I make sure that in training the output of the RNN stays normally distributed. This is accomplished by adding penalization terms in the training which discourage the noise to diverge from the normal distribution.

Low-hanging fruit

I trained on a MacBook Air with an M1 SoC with 16 GB of shared memory for CPU and GPU. Thus, memory was the limiting factor in my experiments.

With more memory, one could

Increase the resolution (currently 64x64 pixels)
Increase the training sequence length (currently 20)
Increase the batch size (currently 32)

A list of papers about point cloud based place recognition, also known as loop closure detection in SLAM (processing)

17 May 16, 2021

FluxTraining.jl gives you an endlessly extensible training loop for deep learning

A flexible neural net training library inspired by fast.ai

86 Dec 31, 2022

FLVIS: Feedback Loop Based Visual Initial SLAM

FLVIS Feedback Loop Based Visual Inertial SLAM 1-Video EuRoC DataSet MH_05 Handheld Test in Lab FlVIS on UAV Platform 2-Relevent Publication: Under Re

182 Dec 4, 2022

Codebase for the Summary Loop paper at ACL2020

Summary Loop This repository contains the code for ACL2020 paper: The Summary Loop: Learning to Write Abstractive Summaries Without Examples. Training

Canny Lab @ The University of California, Berkeley

44 Nov 4, 2022

Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

29 Dec 28, 2022

AirLoop: Lifelong Loop Closure Detection

PyTorch implementation of the method described in the paper VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop.

VoiceLoop PyTorch implementation of the method described in the paper VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop. VoiceLoop is a n

873 Dec 15, 2022

Comments

Images with different input shape
When changing img_size = 64 to count for images with different width and height i.e., also changing the process_img to:

def process_img(file_path, img_w, img_h): img = tf.io.read_file(file_path) img = tf.image.decode_jpeg(img, channels=3) img = tf.image.convert_image_dtype(img, tf.float32) img = tf.image.resize(img, size=(img_h, img_w), method='area') img = tf.image.convert_image_dtype(img, tf.uint8) return img

and the definition of the discriminator to:

def make_discriminator_model(): disc = tf.keras.Sequential( [ layers.Input(shape=(img_h, img_w, 3)), ...

and line 391 to: all_images.append(process_img(item, img_w, img_h))

The training crashes after the first epoch with: ValueError: Cannot reshape a tensor with 49152 elements to shape [32,128] (4096 elements) for '{{node Reshape}} = Reshape[T=DT_FLOAT, Tshape=DT_INT32](discriminator/flatten/Reshape, Reshape/shape)' with input shapes: [32,1536], [2] and with input tensors computed as partial shapes: input[1] = [32,128].

Apparently, there is a problem here: real_first_output = tf.reshape(self.discriminator(images[i,...], training=True), (batch_size, disc_dim)). Somehow, the discriminator is not outputting correct dimension for reshaping. I have doubts that something is wrong with disc_dim but couldn't solve it.

I am using:

img_w = 256 img_h = 144
opened by fm94 1

duralava is a neural network which can simulate a lava lamp in an infinite loop.

Related tags

Overview

duralava

Example

Novelty

How it works

Low-hanging fruit

You might also like...

A list of papers about point cloud based place recognition, also known as loop closure detection in SLAM (processing)

FluxTraining.jl gives you an endlessly extensible training loop for deep learning

FLVIS: Feedback Loop Based Visual Initial SLAM

Codebase for the Summary Loop paper at ACL2020

Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

AirLoop: Lifelong Loop Closure Detection

Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities

Minimalistic PyTorch training loop

PyTorch implementation of the method described in the paper VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop.

Comments

Images with different input shape

Owner

Maximilian Bachl

Fast and Easy Infinite Neural Networks in Python

Code accompanying our paper Feature Learning in Infinite-Width Neural Networks

Use MATLAB to simulate the signal and extract features. Use PyTorch to build and train deep network to do spectrum sensing.

Implements an infinite sum of poisson-weighted convolutions

Pomodoro timer that acknowledges the inexorable, infinite passage of time

Companion code for the paper "An Infinite-Feature Extension for Bayesian ReLU Nets That Fixes Their Asymptotic Overconfidence" (NeurIPS 2021)

Backend code to use MCPI's python API to make infinite worlds with custom generation

Code for "3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop"

Source code for the GPT-2 story generation models in the EMNLP 2020 paper "STORIUM: A Dataset and Evaluation Platform for Human-in-the-Loop Story Generation"