RL-GAN: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

Last update: Nov 10, 2022

Related tags

Deep Learning RL-GAN

Overview

RL-GAN: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

RL-GAN is an official implementation of the paper: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation.

Paper

Shani Gamrian, Yoav Goldberg, "Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation"

@article{DBLP:journals/corr/abs-1806-07377,
  author    = {Shani Gamrian and
               Yoav Goldberg},
  title     = {Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image
               Translation},
  journal   = {CoRR},
  volume    = {abs/1806.07377},
  year      = {2018},
  url       = {http://arxiv.org/abs/1806.07377},
  archivePrefix = {arXiv},
  eprint    = {1806.07377},
  timestamp = {Mon, 13 Aug 2018 16:48:23 +0200},
  biburl    = {https://dblp.org/rec/bib/journals/corr/abs-1806-07377},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

Videos:

Breakout

RoadFighter

Installation

The code was tested on Ubuntu 16.04 with Python 3.6
Install packages by typing the command: pip install -r requirements.txt.
For Road Fighter, clone and install the repo: https://github.com/ShaniGam/retro

Getting Started

Breakout Examples

Train Breakout from scratch:

python -m breakout_a3c.main --num-processes 32 --variation 'standart'

Transfer from standart to diagonals variation and fine-tune the model:

python -m breakout_a3c.main --num-processes 32 --variation diagonals --ft-setting full-ft --test

Collect images for UNIT training:

python -m breakout_a3c.main --collect-images --num-collected-imgs 100000 --variation diagonals --num-processes 1

Train UNIT:

python -m unit.train --trainer UNIT --config unit/configs/breakout-diagonals.yaml

Run Breakout with UNIT:

python -m breakout_a3c.main --variation diagonals --test --ft-setting full-ft --test-gan --gan-dir breakout-diagonals --num-processes 0

Road Fighter Examples

Train level 1 of Road Fighter

python -m roadfighter_a2c.main --num-processes 84

Collect images for UNIT training:

python -m roadfighter_a2c.main -level 1 --collect-images --num-collected-imgs 100000 --num-processes 1
python -m roadfighter_a2c.main -level 2 --collect-images --num-collected-imgs 100000 --num-processes 1

Train UNIT:

python -m unit.train --trainer UNIT --config unit/configs/roadfighter-lvl2.yaml

Run Road Fighter with UNIT:

python -m roadfighter_a2c.main --load --level 2 --test-gan --gan-dir roadfighter-lvl2-kl01 --num-processes 1

Run Road Fighter with UNIT and Imitation Learning:

python -m roadfighter_a2c.main_imitation --load --gan-dir roadfighter-lvl2-kl01 --gan-imitation-file '00320000' --log-name lvl2.log --super-during-rl --level 2 --det-score 5350

Acknowledgments

The code was written by Shani Gamrian and is based on the repositories: pytorch-a3c, pytorch-a2c, UNIT

TO-DO

Add links for pretrained models.
Create videos.

code for our paper "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer"

SHOT++ Code for our TPAMI submission "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer" that is ext

75 Dec 16, 2022

Solving reinforcement learning tasks which require language and vision

Multimodal Reinforcement Learning JAX implementations of the following multimodal reinforcement learning approaches. Dual-coding Episodic Memory from

31 Feb 26, 2022

gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks.

gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks. It is built on top of the OpenAI Gym toolkit.

99 Dec 12, 2022

CompilerGym is a library of easy to use and performant reinforcement learning environments for compiler tasks

721 Jan 3, 2023

"Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback"

This is code repo for our EMNLP 2017 paper "Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback", which implements the A2C algorithm on top of a neural encoder-decoder model and benchmarks the combination under simulated noisy rewards.

131 Oct 21, 2022

Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX

Comments

Unable to reproduce transfer results

I am trying to use this codebase to reproduce the failure of transfer with vanilla RL algorithms. However, I can not reproduce this phenomenon. It seems that this repo does not contain codes for saving trained models and the training accuracy is always zero on Breakout. I wonder if this is the original code the authors use. Also, I tried to use stable-baselines with wrappers in this repo, however, it seems that vanilla algorithms can successfully transfer to perturbed environments.

I wonder if there is something I missed since the results are contradictory to the paper.

opened by MouseHu 1
Would it be possible to receive pretrained models for the a3c for breakout and a2c for roadfighter?

Hi,

I'm trying to implement the code for the breakout environment, unfortunately it seems to take too long on the computational resources I have access to. Would it be possible to receive the pre-trained models for the RL?

opened by PeteXC 0

RL-GAN: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

Related tags

Overview

RL-GAN: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

Paper

Videos:

Installation

Getting Started

Breakout Examples

Road Fighter Examples

Acknowledgments

TO-DO

You might also like...

code for our paper "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer"

Solving reinforcement learning tasks which require language and vision

gym-anm is a framework for designing reinforcement learning (RL) environments that model Active Network Management (ANM) tasks in electricity distribution networks.

CompilerGym is a library of easy to use and performant reinforcement learning environments for compiler tasks

"Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback"

Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX

Reinforcement-learning - Repository of the class assignment questions for the course on reinforcement learning

Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

Related resources for our EMNLP 2021 paper Plan-then-Generate: Controlled Data-to-Text Generation via Planning

Comments

Unable to reproduce transfer results

Would it be possible to receive pretrained models for the a3c for breakout and a2c for roadfighter?

Owner

Code of Adverse Weather Image Translation with Asymmetric and Uncertainty aware GAN

docTR by Mindee (Document Text Recognition) - a seamless, high-performing & accessible library for OCR-related tasks powered by Deep Learning.

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

CVPR 2021: "The Spatially-Correlative Loss for Various Image Translation Tasks"

Transfer style api - An API to use with Tranfer Style App, where you can use two image and transfer the style

a morph transfer UGATIT for image translation.

A curated list of awesome resources related to Semantic Search🔎 and Semantic Similarity tasks.

Transfer-Learn is an open-source and well-documented library for Transfer Learning.

Official code for the paper "Why Do Self-Supervised Models Transfer? Investigating the Impact of Invariance on Downstream Tasks".

Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".