Recurrent Variational Autoencoder that generates sequential data implemented with pytorch

Related tags

Deep Learning python nlp deep-learning pytorch vae
Overview

Pytorch Recurrent Variational Autoencoder

Model:

This is the implementation of Samuel Bowman's Generating Sentences from a Continuous Space with Kim's Character-Aware Neural Language Models embedding for tokens

Sampling examples:

the new machine could be used to increase the number of ventures block in the company 's <unk> shopping system to finance diversified organizations

u.s. government officials also said they would be willing to consider whether the proposal could be used as urging and programs

men believe they had to go on the <unk> because their <unk> were <unk> expensive important

the companies insisted that the color set could be included in the program

Usage

Before model training it is necessary to train word embeddings:

$ python train_word_embeddings.py

This script train word embeddings defined in Mikolov et al. Distributed Representations of Words and Phrases

Parameters:

--use-cuda

--num-iterations

--batch-size

--num-sample –– number of sampled from noise tokens

To train model use:

$ python train.py

Parameters:

--use-cuda

--num-iterations

--batch-size

--learning-rate

--dropout –– probability of units to be zeroed in decoder input

--use-trained –– use trained before model

To sample data after training use:

$ python sample.py

Parameters:

--use-cuda

--num-sample

Comments
  • Error while running train_word_embeddings.py

    Error while running train_word_embeddings.py

    File "/pytorch_RVAE-master/utils/functional.py", line 2, in fold return a if (len(l) == 0) else fold(f, l[1:], f(a, l[0])) File "/pytorch_RVAE-master/utils/functional.py", line 2, in fold return a if (len(l) == 0) else fold(f, l[1:], f(a, l[0])) File "/pytorch_RVAE-master/utils/functional.py", line 6, in f_and return (x and y) RuntimeError: bool value of Tensor with more than one value is ambiguous

    opened by SummmerSnow 3
  • RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)

    RuntimeError: dimension out of range (expected to be in range of [-2, 1], but got 2)

    Hi! I just ran python train.py and received this traceback:

    Traceback (most recent call last):
  File "train.py", line 59, in <module>
    cross_entropy, kld, coef = train_step(iteration, args.batch_size, args.use_cuda, args.dropout)
  File "/home/****/pytorch-experiments/insane_playgorund/pytorch_RVAE/model/rvae.py", line 104, in train
    z=None)
  File "/home/****/.virtualenvs/pytorch-env/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/****/pytorch-experiments/insane_playgorund/pytorch_RVAE/model/rvae.py", line 64, in forward
    encoder_input = self.embedding(encoder_word_input, encoder_character_input)
  File "/home/****/.virtualenvs/pytorch-env/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/****/pytorch-experiments/insane_playgorund/pytorch_RVAE/selfModules/embedding.py", line 47, in forward
    character_input = self.TDNN(character_input)
  File "/home/****/.virtualenvs/pytorch-env/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/****/pytorch-experiments/insane_playgorund/pytorch_RVAE/selfModules/tdnn.py", line 42, in forward
    xs = [x.max(2)[0].squeeze(2) for x in xs]
  File "/home/****/pytorch-experiments/insane_playgorund/pytorch_RVAE/selfModules/tdnn.py", line 42, in <listcomp>
    xs = [x.max(2)[0].squeeze(2) for x in xs]
  File "/home/****/..virtualenvs/pytorch-env/lib/python3.5/site-packages/torch/autograd/variable.py", line 750, in squeeze
    return Squeeze.apply(self, dim)
  File "/home/****/.virtualenvs/pytorch-env/lib/python3.5/site-packages/torch/autograd/_functions/tensor.py", line 378, in forward
    result = input.squeeze(dim)

    Quick glance can't locate the source of the problem. Python 3.5 and Pytorch 0.2.0_3

    opened by mojesty 3
  • About the loss formula

    About the loss formula

    In rvae.py "loss = 79 * cross_entropy + kld_coef(i) * kld". Could you please explain why it is multiplied by 79 and what's the meaning of kld_coef(i)?

    opened by xushenkun 2
  • allow_pickle=False problem in train_word_embeddings.py

    allow_pickle=False problem in train_word_embeddings.py

    Hi, I really like this code. When I try to train word embeddings, it shows this error. Does anyone have the same problem and have solved it? I changed CUDA default to False since I am running on MacOS (10.14). I run python 3.6.1 on conda and the following modules: certifi==2019.3.9 cffi==1.12.3 mkl-fft==1.0.10 mkl-random==1.0.2 numpy==1.16.3 olefile==0.46 Pillow==4.2.1 pycparser==2.19 six==1.12.0 torch==1.0.1.post2 torchvision==0.2.2

    Traceback (most recent call last): File "train_word_embeddings.py", line 25, in batch_loader = BatchLoader('') File "/Users/davis/Downloads/Projects/side-projects/pytorch_RVAE/utils/batch_loader.py", line 104, in init self.tensor_files) File "/Users/davis/Downloads/Projects/side-projects/pytorch_RVAE/utils/batch_loader.py", line 224, in load_preprocessed for input_type in tensor_files] File "/Users/davis/Downloads/Projects/side-projects/pytorch_RVAE/utils/batch_loader.py", line 224, in for input_type in tensor_files] File "/Users/davis/Downloads/Projects/side-projects/pytorch_RVAE/utils/batch_loader.py", line 223, in [self.word_tensor, self.character_tensor] = [np.array([np.load(target) for target in input_type]) File "/Users/davis/miniconda3/envs/pytorchRVAE/lib/python3.6/site-packages/numpy/lib/npyio.py", line 447, in load pickle_kwargs=pickle_kwargs) File "/Users/davis/miniconda3/envs/pytorchRVAE/lib/python3.6/site-packages/numpy/lib/format.py", line 692, in read_array raise ValueError("Object arrays cannot be loaded when " ValueError: Object arrays cannot be loaded when allow_pickle=False

    opened by davislf2 1
  • Error while running train.py

    Error while running train.py

    Traceback (most recent call last): File "train.py", line 59, in cross_entropy, kld, coef = train_step(iteration, args.batch_size, args.use_cuda, args.dropout) File "/home/tonygrey/pytorch_RVAE/model/rvae.py", line 113, in train loss.backward() File "/home/tonygrey/miniconda3/lib/python3.6/site-packages/torch/autograd/variable.py", line 167, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables) File "/home/tonygrey/miniconda3/lib/python3.6/site-packages/torch/autograd/init.py", line 99, in backward variables, grad_variables, retain_graph) RuntimeError: invalid argument 1: the number of sizes provided must be greater or equal to the number of dimensions in the tensor at /opt/conda/conda-bld/pytorch_1518243271935/work/torch/lib/THC/generic/THCTensor.c:326

    opened by ghost 0
  • How to create MSCOCO dataset

    How to create MSCOCO dataset

    Hi, I got the MSCOCO captions_train2014.json and captions_val2014.json, as described in the paper, there are 82,783 train samples and 40,504 val samples, every sample contains 5 captions. If I omit one caption and combine the other four into two paraphrase pairs, there will be about 2*(82,783 + 40,504)=246,574 pairs. How can i get the 320k paraphrase pairs?

    opened by Frankey419 1
  • Error: bool value of Tensor with more than one value is ambiguous

    Error: bool value of Tensor with more than one value is ambiguous

    File "/home/sarvesh23/pytorch_RVAE/utils/functional.py", line 6, in f_and return x and y RuntimeError: bool value of Tensor with more than one value is ambiguous

    I am running train_word_embeddings.py. Any hint what I am doing wrong ?

    opened by sarvesh710 5
  • KLD in Loss function

    KLD in Loss function

    Hi, would you tell me why the kld calculation only involves what it seems to be the encoder or approximated (to the posterior) function? Isn't it calculated base on 2 distributions?

            mu = self.context_to_mu(context)
        logvar = self.context_to_logvar(context) # to z sampled from
        std = t.exp(0.5 * logvar)
        z = Variable(t.randn([batch_size, self.params.latent_variable_size]))
        if use_cuda:
            z = z.cuda()
        z = z * std + mu
        kld = (-0.5 * t.sum(logvar - t.pow(mu, 2) - t.exp(logvar) + 1, 1)).mean().squeeze()
    opened by gissemari 0
  • RuntimeError: bool value of Variable objects containing non-empty torch.cuda.FloatTensor is ambiguous

    RuntimeError: bool value of Variable objects containing non-empty torch.cuda.FloatTensor is ambiguous

    mldl@ub1604:~/ub16_prj/pytorch_RVAE$ python3 train_word_embeddings.py preprocessed data was found and loaded Traceback (most recent call last): File "train_word_embeddings.py", line 47, in out = neg_loss(input, target, args.num_sample).mean() File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 224, in call result = self.forward(*input, **kwargs) File "/home/mldl/ub16_prj/pytorch_RVAE/selfModules/neg.py", line 38, in forward assert parameters_allocation_check(self),
    File "/home/mldl/ub16_prj/pytorch_RVAE/utils/functional.py", line 15, in parameters_allocation_check return fold(f_and, parameters, True) or not fold(f_or, parameters, False) File "/home/mldl/ub16_prj/pytorch_RVAE/utils/functional.py", line 2, in fold return a if (len(l) == 0) else fold(f, l[1:], f(a, l[0])) File "/home/mldl/ub16_prj/pytorch_RVAE/utils/functional.py", line 2, in fold return a if (len(l) == 0) else fold(f, l[1:], f(a, l[0])) File "/home/mldl/ub16_prj/pytorch_RVAE/utils/functional.py", line 6, in f_and return x and y File "/usr/local/lib/python3.5/dist-packages/torch/autograd/variable.py", line 123, in bool torch.typename(self.data) + " is ambiguous") RuntimeError: bool value of Variable objects containing non-empty torch.cuda.FloatTensor is ambiguous mldl@ub1604:~/ub16_prj/pytorch_RVAE$

    opened by loveJasmine 1
  • Loss Function

    Loss Function

    Hi,

    Would you like to tell me why you added scalar '79' in front of the cross entropy (RVAE, Line 110)? loss = 79 * cross_entropy + kld_coef(i) * kld

    best

    opened by dongqian0206 2
Owner
Daniil Gavrilov
The Last AI Bender
Daniil Gavrilov
GitHub
PyTorch Autoencoders - Implementing a Variational Autoencoder (VAE) Series in Pytorch.

PyTorch Autoencoders Implementing a Variational Autoencoder (VAE) Series in Pytorch. Inspired by this repository Model List check model paper conferen

Subin An 8 Nov 21, 2022
CoSMA: Convolutional Semi-Regular Mesh Autoencoder. From Paper "Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes"

Mesh Convolutional Autoencoder for Semi-Regular Meshes of Different Sizes Implementation of CoSMA: Convolutional Semi-Regular Mesh Autoencoder arXiv p

Fraunhofer SCAI 10 Oct 11, 2022
Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

VariationalRecurrentNeuralNetwork Pytorch implementation of the Variational RNN (VRNN), from A Recurrent Latent Variable Model for Sequential Data. Th

emmanuel 251 Dec 17, 2022
Clockwork Variational Autoencoder

Clockwork Variational Autoencoders (CW-VAE) Vaibhav Saxena, Jimmy Ba, Danijar Hafner If you find this code useful, please reference in your paper: @ar

Vaibhav Saxena 35 Nov 6, 2022
VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

VITS: Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech Jaehyeon Kim, Jungil Kong, and Juhee Son In our rece

Jaehyeon Kim 1.7k Jan 8, 2023
Implementation for "Manga Filling Style Conversion with Screentone Variational Autoencoder" (SIGGRAPH ASIA 2020 issue)

Manga Filling with ScreenVAE SIGGRAPH ASIA 2020 | Project Website | BibTex This repository is for ScreenVAE introduced in the following paper "Manga F

null 30 Dec 24, 2022
Variational autoencoder for anime face reconstruction

VAE animeface Variational autoencoder for anime face reconstruction Introduction This repository is an exploratory example to train a variational auto

Minzhe Zhang 2 Dec 11, 2021
Code of 3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces

3D Shape Variational Autoencoder Latent Disentanglement via Mini-Batch Feature Swapping for Bodies and Faces Installation After cloning the repo open

null 37 Dec 3, 2022
Official implementation of the RAVE model: a Realtime Audio Variational autoEncoder

RAVE: Realtime Audio Variational autoEncoder Official implementation of RAVE: A variational autoencoder for fast and high-quality neural audio synthes

ACIDS 587 Jan 1, 2023
Implementation of Bidirectional Recurrent Independent Mechanisms (Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules)

BRIMs Bidirectional Recurrent Independent Mechanisms Implementation of the paper Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neura

Sarthak Mittal 26 May 26, 2022
Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data

LiDAR-MOS: Moving Object Segmentation in 3D LiDAR Data This repo contains the code for our paper: Moving Object Segmentation in 3D LiDAR Data: A Learn

Photogrammetry & Robotics Bonn 394 Dec 29, 2022
Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Project looking into use of autoencoder for semi-supervised learning and comparing data requirements compared to supervised learning.

Tom-R.T.Kvalvaag 2 Dec 17, 2021
This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et al. 2020

README This is the official Pytorch implementation of "Lung Segmentation from Chest X-rays using Variational Data Imputation", Raghavendra Selvan et a

Raghav 42 Dec 15, 2022
MADE (Masked Autoencoder Density Estimation) implementation in PyTorch

pytorch-made This code is an implementation of "Masked AutoEncoder for Density Estimation" by Germain et al., 2015. The core idea is that you can turn

Andrej 498 Dec 30, 2022
Molecular AutoEncoder in PyTorch

MolEncoder Molecular AutoEncoder in PyTorch Install $ git clone https://github.com/cxhernandez/molencoder.git && cd molencoder $ python setup.py insta

Carlos Hernández 80 Dec 5, 2022
Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

null 730 Jan 9, 2023
Code for the ECCV2020 paper "A Differentiable Recurrent Surface for Asynchronous Event-Based Data"

A Differentiable Recurrent Surface for Asynchronous Event-Based Data Code for the ECCV2020 paper "A Differentiable Recurrent Surface for Asynchronous

Marco Cannici 21 Oct 5, 2022
A real world application of a Recurrent Neural Network on a binary classification of time series data

What is this This is a real world application of a Recurrent Neural Network on a binary classification of time series data. This project includes data

Josep Maria Salvia Hornos 2 Jan 30, 2022
PyTorch implementation DRO: Deep Recurrent Optimizer for Structure-from-Motion

DRO: Deep Recurrent Optimizer for Structure-from-Motion This is the official PyTorch implementation code for DRO-sfm. For technical details, please re

Alibaba Cloud 56 Dec 12, 2022