MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space

Last update: Dec 15, 2022

Related tags

Deep Learning modals

Overview

Update (20 Jan 2020): MODALS on text data is avialable

MODALS

MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space

Introduction
Getting Started
Run Search
Run Training
Citation

Introduction

MODALS is a framework to apply automated data augmentation to augment data for any modality in a generic way. It exploits automated data augmentation to fine-tune four universal data transformation operations in the latent space to adapt the transform to data of different modalities.

This repository contains code for the work "MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space" (https://openreview.net/pdf?id=XjYgR6gbCEc) implemented using the PyTorch library. It includes searching and training of the SST2 and TREC6 datasets.

Getting Started

Code supports Python 3.

Install requirements

pip install -r requirements.txt

Setting up directory path

In modals/setup.py, specify the dataset path for DATA_DIR and the path to the directory that contains the glove embeddings for EMB_DIR.

Run MODALS search

Script to search for the augmentation policy for SST2 and TREC6 datasets is located in scripts/search.sh. Pass the dataset name as the arguement to call the script.

For example, to search for the augmentation policy for SST2 dataset:

bash scripts/search.sh sst2

The training log and candidate policies of the search will be output to the ./ray_experiments directory.

Run MODALS training

Two searched policy is included in the ./schedule directory. The script to apply the searched policy for training SST2 and TREC6 is located in scripts/train.sh. Pass the dataset name as the arguement to call the script.

bash scripts/train.sh sst2

Citation

If you use MODALS in your research, please cite:

@inproceedings{cheung2021modals,
  title     =  {{\{}MODALS{\}}: Modality-agnostic Automated Data Augmentation in the Latent Space},
  author    =  {Tsz-Him Cheung and Dit-Yan Yeung},
  booktitle =  {International Conference on Learning Representations},
  year      =  {2021},
  url       =  {https://openreview.net/forum?id=XjYgR6gbCEc}
}

You might also like...

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models Code accompanying CVPR'20 paper of the same title. Paper lin

7k Dec 30, 2022

Feed forward VQGAN-CLIP model, where the goal is to eliminate the need for optimizing the latent space of VQGAN for each input prompt

Feed forward VQGAN-CLIP model, where the goal is to eliminate the need for optimizing the latent space of VQGAN for each input prompt. This is done by

135 Dec 30, 2022

PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)

Authors official PyTorch implementation of the "WarpedGANSpace: Finding non-linear RBF paths in GAN latent space" [ICCV 2021].

100 Dec 6, 2022

InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing

InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing Figure: High-quality facial attributes editing results with InterFaceGA

GenForce: May Generative Force Be with You

1.3k Jan 9, 2023

Code for "SRHEN: Stepwise-Refining Homography Estimation Network via Parsing Geometric Correspondences in Deep Latent Space"

SRHEN This is a better and simpler implementation for "SRHEN: Stepwise-Refining Homography Estimation Network via Parsing Geometric Correspondences in

1 Oct 28, 2022

Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

Tensor Component Analysis for Interpreting the Latent Space of GANs [ paper | project page ] Code to reproduce the results in the paper "Tensor Compon

4 Jun 17, 2022

Visualizer using audio and semantic analysis to explore BigGAN (Brock et al., 2018) latent space.

BigGAN Audio Visualizer Description This visualizer explores BigGAN (Brock et al., 2018) latent space by using pitch/tempo of an audio file to generat

2 Nov 21, 2022

Implementation based on Paper - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

3 Jul 8, 2022

《LXMERT: Learning Cross-Modality Encoder Representations from Transformers》(EMNLP 2020)

The Most Important Thing. Our code is developed based on: LXMERT: Learning Cross-Modality Encoder Representations from Transformers

53 Dec 16, 2022

Comments

zipfile.BadZipFile: File is not a zip file

Can you guide me to solve the issue please?

I get this error "zipfile.BadZipFile: File is not a zip file" in this part of the code file:data_utils.py, line:1896 TEXT.build_vocab(train, vectors=GloVe(name='6B', dim=300, cache=EMB_DIR))

opened by mmoha014 1

MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space

Related tags

Overview

MODALS

Table of Contents

Introduction

Getting Started

Install requirements

Setting up directory path

Run MODALS search

Run MODALS training

Citation

You might also like...

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

Feed forward VQGAN-CLIP model, where the goal is to eliminate the need for optimizing the latent space of VQGAN for each input prompt

PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)

InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing

Code for "SRHEN: Stepwise-Refining Homography Estimation Network via Parsing Geometric Correspondences in Deep Latent Space"

Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

Visualizer using audio and semantic analysis to explore BigGAN (Brock et al., 2018) latent space.

Implementation based on Paper - Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

《LXMERT: Learning Cross-Modality Encoder Representations from Transformers》(EMNLP 2020)

Comments

zipfile.BadZipFile: File is not a zip file

Owner

MARS: Learning Modality-Agnostic Representation for Scalable Cross-media Retrieva

An integration of several popular automatic augmentation methods, including OHL (Online Hyper-Parameter Learning for Auto-Augmentation Strategy) and AWS (Improving Auto Augment via Augmentation Wise Weight Sharing) by Sensetime Research.

The source code for the Cutoff data augmentation approach proposed in this paper: "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation".

Image transformations designed for Scene Text Recognition (STR) data augmentation. Published at ICCV 2021 Workshop on Interactive Labeling and Data Augmentation for Vision.

Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Search

Face Identity Disentanglement via Latent Space Mapping [SIGGRAPH ASIA 2020]

Navigating StyleGAN2 w latent space using CLIP

Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

Disentangled Face Attribute Editing via Instance-Aware Latent Space Search, accepted by IJCAI 2021.

[CVPR 2020] Interpreting the Latent Space of GANs for Semantic Face Editing