Self-attentive task GAN for space domain awareness data augmentation.

Related tags

Deep Learning gan attention task-gan
Overview

SATGAN

TODO: update the article URL once published.

Article about this implemention

The self-attentive task generative adversarial network (SATGAN) learns to emulate realistic target sensor noise characteristics in order to augment existing datasets with simulated scenes that better approximate real-world systems. It learns a mapping from random input noise to realistic target-domain sensor characteristics while maintaining semantic information in simulated scenes through the use of a task network. Example real images of a space domain awareness (SDA) scene from the original paper are shown below:

Real images

Example noiseless simulated scenes used as context are below:

Context images

Finally example simulated scenes with generated addative noise are shown below:

Fake images

SATGAN comprises three parts: a generator based on a U-net implementation, a discriminator based on PatchGAN, and a task network based on [Fletcher et al.]. The SATGAN architecture is illustrated below:

SATGAN architecture

Setup

Prerequisites

  • Tensorflow >= 2.2.1
  • Tensorflow-addons >= 0.11.2 (for optional mish activation)
  • MISS YOLOv3

Recommended

  • Linux with Tensorflow GPU edition + cuDNN

Getting Started

# clone this repo
git clone https://github.com/Engineero/satgan.git
cd satgan

# train the model (this may take 1-8 hours depending on GPU, on CPU you will be waiting for a bit)
python train_satgan.py \
  --mode train \
  --output_dir model_train \
  --max_epochs 200 \
  --input_dir my_data/train \

Citation

TODO: update paper link

If you use this code for your research, please cite the paper this code is based on: Self-attending task generative adversarial network for realistic satellite image creation:

@article{toner_self-attending_2021,
	title = {Self-{Attending} {Task} {Generative} {Adversarial} {Network} for {Realistic} {Satellite} {Image} {Creation}},
	url = {https://arxiv.org/abs/2111.09463v1},
	language = {en},
	urldate = {2021-11-19},
	author = {Toner, Nathan and Fletcher, Justin},
	month = nov,
	year = {2021},
	file = {Snapshot:/Users/nathantoner/Zotero/storage/K7AHTQEU/2111.html:text/html},
}

Acknowledgments

You might also like...
“Data Augmentation for Cross-Domain Named Entity Recognition” (EMNLP 2021)
“Data Augmentation for Cross-Domain Named Entity Recognition” (EMNLP 2021)

Data Augmentation for Cross-Domain Named Entity Recognition Authors: Shuguang Chen, Gustavo Aguilar, Leonardo Neves and Thamar Solorio This repository

RiTUAL@UH 18 Sep 10, 2022
DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time
DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time Introduction This is official implementation for DR-GAN (IEEE TCS

Kang Liao 18 Dec 23, 2022
PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)
PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)

Authors official PyTorch implementation of the "WarpedGANSpace: Finding non-linear RBF paths in GAN latent space" [ICCV 2021].

Christos Tzelepis 100 Dec 6, 2022
[CVPR 2022] TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing
[CVPR 2022] TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing

TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing (CVPR 2022) This repository provides the official PyTorch impleme

Billy XU 128 Jan 3, 2023
Space robot - (Course Project) Using the space robot to capture the target satellite that is disabled and spinning, then stabilize and fix it up

Space robot - (Course Project) Using the space robot to capture the target satellite that is disabled and spinning, then stabilize and fix it up

Mingrui Yu 3 Jan 7, 2022
Pytorch codes for
Pytorch codes for "Self-supervised Multi-view Stereo via Effective Co-Segmentation and Data-Augmentation"

Self-Supervised-MVS This repository is the official PyTorch implementation of our AAAI 2021 paper: "Self-supervised Multi-view Stereo via Effective Co

hongbin_xu 127 Jan 4, 2023
A framework for attentive explainable deep learning on tabular data

🧠 kendrite A framework for attentive explainable deep learning on tabular data 💨 Quick start kedro run 🧱 Built upon Technology Description Links ke

Marnix Koops 3 Nov 6, 2021
Code release for
Code release for "Transferable Semantic Augmentation for Domain Adaptation" (CVPR 2021)

Transferable Semantic Augmentation for Domain Adaptation Code release for "Transferable Semantic Augmentation for Domain Adaptation" (CVPR 2021) Paper

null 66 Dec 16, 2022
UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus
UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus

UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus General info This is

null 71 Oct 25, 2022
Comments
  • Merge updates for running with TF2 YOLO task net

    Merge updates for running with TF2 YOLO task net

    The develop branch is a pretty major overhaul, but the goal of it all is just to run the same script with updated MISS YOLO scripts. Some bugs came up (see issues tracker) in this process, so I'm creating a merge request to also help track changes to the dev branch for this purpose.

    enhancement 
    opened by Engineero 3
  • Fix task net inference with TF2 version of MISS YOLO

    Fix task net inference with TF2 version of MISS YOLO

    I created a feature branch to track work on fixing the task network inference when using the TF2 (latest) version of MISS YOLO as the task network. This way we can nuke whatever we need to here without compromising whatever is working in the develop branch.

    bug help wanted 
    opened by Engineero 1
  • Training with TF2 (latest) YOLO task net results in strange task net behavior, results

    Training with TF2 (latest) YOLO task net results in strange task net behavior, results

    When training the network with the older version of MISS YOLO trained in TF1.x, network trains fine and resulting artificial images are OK, not great.

    When training with the latest version of MISS YOLO trained in TF2.x, the task network always fails to find objects in the scene, even in real SatNet images, which doesn't make any sense given how that same network performs on SatNet images when evaluating using MISS evaluation scripts.

    The fact that the network never finds real objects in the scenes when training here indicates to me that there is still a bug somewhere in my code, or in the way I am handling inputs from TF2.x version of YOLO. I haven't been able to find it. I'm creating this issue to track progress towards fixing this.

    bug help wanted 
    opened by Engineero 0
Owner
Nathan
:(){ : | :& };:
Nathan
GitHub
the official code for ICRA 2021 Paper: "Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation"

G2S This is the official code for ICRA 2021 Paper: Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation by Hemang

NeurAI 4 Jul 27, 2022
The source code for the Cutoff data augmentation approach proposed in this paper: "A Simple but Tough-to-Beat Data Augmentation Approach for Natural Language Understanding and Generation".

Cutoff: A Simple Data Augmentation Approach for Natural Language This repository contains source code necessary to reproduce the results presented in

Dinghan Shen 49 Dec 22, 2022
Image transformations designed for Scene Text Recognition (STR) data augmentation. Published at ICCV 2021 Workshop on Interactive Labeling and Data Augmentation for Vision.

Data Augmentation for Scene Text Recognition (ICCV 2021 Workshop) (Pronounced as "strog") Paper Arxiv Why it matters? Scene Text Recognition (STR) req

Rowel Atienza 152 Dec 28, 2022
[NeurIPS 2021] Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data

Deceive D: Adaptive Pseudo Augmentation for GAN Training with Limited Data (NeurIPS 2021) This repository provides the official PyTorch implementation

Liming Jiang 155 Nov 30, 2021
MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space

Update (20 Jan 2020): MODALS on text data is avialable MODALS MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space Table of Conte

null 38 Dec 15, 2022
A Structured Self-attentive Sentence Embedding

Structured Self-attentive sentence embeddings Implementation for the paper A Structured Self-Attentive Sentence Embedding, which was published in ICLR

Kaushal Shetty 488 Nov 28, 2022
Locally Constrained Self-Attentive Sequential Recommendation

LOCKER This is the pytorch implementation of this paper: Locally Constrained Self-Attentive Sequential Recommendation. Zhankui He, Handong Zhao, Zhe L

Zhankui (Aaron) He 8 Jul 30, 2022
A PaddlePaddle implementation of Time Interval Aware Self-Attentive Sequential Recommendation.

TiSASRec.paddle A PaddlePaddle implementation of Time Interval Aware Self-Attentive Sequential Recommendation. Introduction 论文：Time Interval Aware Sel

Paddorch 2 Nov 28, 2021
Implementation for "Seamless Manga Inpainting with Semantics Awareness" (SIGGRAPH 2021 issue)

Seamless Manga Inpainting with Semantics Awareness [SIGGRAPH 2021](To appear) | Project Website | BibTex Introduction: Manga inpainting fills up the d

null 101 Jan 1, 2023
Pytorch implementation of Learning with Opponent-Learning Awareness

Pytorch implementation of Learning with Opponent-Learning Awareness using DiCE

Alexis David Jacq 82 Sep 15, 2022