Contrastively Disentangled Sequential Variational Audoencoder

Related tags

Deep Learning C-DSVAE
Overview

Contrastively Disentangled Sequential Variational Audoencoder (C-DSVAE)

Overview

This is the implementation for our C-DSVAE, a novel self-supervised disentangled sequential representation learning method.

Requirements

  • Python 3
  • PyTorch 1.7
  • Numpy 1.18.5

Dataset

Sprites

We provide the raw Sprites .npy files. One can also find the dataset on a third-party repo.

For each split (train/test), we expect the following components for each sequence sample

  • x: raw sample of shape [8, 3, 64, 64]
  • c_aug: content augmentation of shape [8, 3, 64, 64]
  • m_aug: motion augmentation of shape [8, 3, 64, 64]
  • motion factors: action (3 classes), direction (3 classes)
  • content factors: skin, tops, pants, hair (each with 6 classes)

Running

Train

./run_cdsvae.sh

Test

./run_test_sprite.sh

Classification Judge

The judge classifiers are pretrained with full supervision separately.

C-DSVAE Checkpoints

We provide a sample Sprites checkpoint. Checkpoint parameters can be found in ./run_test_sprite.sh.

Paper

If you are inspired by our work, please cite the following paper:

@inproceedings{bai2021contrastively,
  title={Contrastively Disentangled Sequential Variational Autoencoder},
  author={Bai, Junwen and Wang, Weiran and Gomes, Carla},
  booktitle={Advances in Neural Information Processing Systems},
  volume={},
  year={2021}
}
You might also like...
Implementation of the method described in the Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.
Implementation of the method described in the Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Implementation of the method described in the Speech Resynthesis from Di

null 4 Mar 11, 2022
[arXiv22] Disentangled Representation Learning for Text-Video Retrieval
[arXiv22] Disentangled Representation Learning for Text-Video Retrieval

Disentangled Representation Learning for Text-Video Retrieval This is a PyTorch implementation of the paper Disentangled Representation Learning for T

Qiang Wang 49 Dec 18, 2022
Official PyTorch implementation of BlobGAN: Spatially Disentangled Scene Representations

BlobGAN: Spatially Disentangled Scene Representations Official PyTorch Implementation Paper | Project Page | Video | Interactive Demo BlobGAN.mp4 This

null 148 Dec 29, 2022
Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks
Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks

CyGNet This repository reproduces the AAAI'21 paper “Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Network

CunchaoZ 89 Jan 3, 2023
Sequential Model-based Algorithm Configuration

SMAC v3 Project Copyright (C) 2016-2018 AutoML Group Attention: This package is a reimplementation of the original SMAC tool (see reference below). Ho

AutoML-Freiburg-Hannover 778 Jan 5, 2023
Sequential model-based optimization with a `scipy.optimize` interface
Sequential model-based optimization with a `scipy.optimize` interface

Scikit-Optimize Scikit-Optimize, or skopt, is a simple and efficient library to minimize (very) expensive and noisy black-box functions. It implements

Scikit-Optimize 2.5k Jan 4, 2023
Leveraging Two Types of Global Graph for Sequential Fashion Recommendation, ICMR 2021

This is the repo for the paper: Leveraging Two Types of Global Graph for Sequential Fashion Recommendation Requirements OS: Ubuntu 16.04 or higher ver

Yujuan Ding 10 Oct 10, 2022
Weighing Counts: Sequential Crowd Counting by Reinforcement Learning
Weighing Counts: Sequential Crowd Counting by Reinforcement Learning

LibraNet This repository includes the official implementation of LibraNet for crowd counting, presented in our paper: Weighing Counts: Sequential Crow

Hao Lu 18 Nov 5, 2022
StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking

StackRec: Efficient Training of Very Deep Sequential Recommender Models by Iterative Stacking Datasets You can download datasets that have been pre-pr

null 25 May 29, 2022
Comments
  • '../dataset/Sprite/data.pkl' is not found.

    '../dataset/Sprite/data.pkl' is not found.

    Thank you for sharing your programs. I attempted to reproduce C-DSVAE but couldn't find the edited dataset(data.pkl). If you might see #1, Could you share the dataset or the program to create it?

    opened by littletake 2
  • About time variant latent z_rnn

    About time variant latent z_rnn

    Hello, I have a question about the generation of time variant latent z_t

    I think in the paper z_t depends on z_0,...,z_t-1.

    q(z_t|z_{<t}) = ...

    while in your code latents are sampled independently, although with a rnn hidden state.

    features, _ = self.z_rnn(lstm_out)
z_mean = self.z_mean(features)
z_logvar = self.z_logvar(features)
z_post = self.reparameterize(z_mean, z_logvar, random_sampling=True)

    I think the randomness of previous latent are not introduced to the cureent latent, I'm wondering whether this is the same meaning with the paper?

    opened by heyuanYao-pku 1
Owner
Junwen Bai
A machine learning researcher.
Junwen Bai
GitHub
This is an official implementation of our CVPR 2021 paper "Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression" (https://arxiv.org/abs/2104.02300)

Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression Introduction In this paper, we are interested in the bottom-up paradigm of estima

HRNet 367 Dec 27, 2022
Disentangled Cycle Consistency for Highly-realistic Virtual Try-On, CVPR 2021

Disentangled Cycle Consistency for Highly-realistic Virtual Try-On, CVPR 2021 [WIP] The code for CVPR 2021 paper 'Disentangled Cycle Consistency for H

ChongjianGE 94 Dec 11, 2022
DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control

DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control One version of our system is implemented using the

null 260 Nov 28, 2022
Disentangled Face Attribute Editing via Instance-Aware Latent Space Search, accepted by IJCAI 2021.

Instance-Aware Latent-Space Search This is a PyTorch implementation of the following paper: Disentangled Face Attribute Editing via Instance-Aware Lat

null 67 Dec 21, 2022
PyTorch implementation of: Michieli U. and Zanuttigh P., "Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations", CVPR 2021.

Continual Semantic Segmentation via Repulsion-Attraction of Sparse and Disentangled Latent Representations This is the official PyTorch implementation

Multimedia Technology and Telecommunication Lab 42 Nov 9, 2022
Code for CVPR2021 paper 'Where and What? Examining Interpretable Disentangled Representations'.

PS-SC GAN This repository contains the main code for training a PS-SC GAN (a GAN implemented with the Perceptual Simplicity and Spatial Constriction c

Xinqi/Steven Zhu 40 Dec 16, 2022
Implementation of StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation in PyTorch

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation Implementation of StyleSpace Analysis: Disentangled Controls for StyleGAN Ima

Xuanchi Ren 86 Dec 7, 2022
Disentangled Lifespan Face Synthesis

Disentangled Lifespan Face Synthesis Project Page | Paper Demo on Colab Preparation Please follow this github to prepare the environments and dataset.

何森 50 Sep 20, 2022
An official reimplementation of the method described in the INTERSPEECH 2021 paper - Speech Resynthesis from Discrete Disentangled Self-Supervised Representations.

Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Implementation of the method described in the Speech Resynthesis from Di

Facebook Research 253 Jan 6, 2023
A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation (ICCV 2021)

A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation (ICCV 2021) This repository contains the official implemen

null 81 Dec 14, 2022