Auto-DrAC: Automatic Data-Regularized Actor-Critic
This is a PyTorch implementation of the methods proposed in
Automatic Data Augmentation for Generalization in Deep Reinforcement Learning by
Roberta Raileanu, Max Goldstein, Denis Yarats, Ilya Kostrikov, and Rob Fergus.
Citation
If you use this code in your own work, please cite our paper:
@article{raileanu2020automatic,
title={Automatic Data Augmentation for Generalization in Deep Reinforcement Learning},
author={Raileanu, Roberta and Goldstein, Max and Yarats, Denis and Kostrikov, Ilya and Fergus, Rob},
journal={arXiv preprint arXiv:2006.12862},
year={2020}
}
Requirements
The code was run on a GPU with CUDA 10.2. To install all the required dependencies:
conda create -n auto-drac python=3.7
conda activate auto-drac
git clone [email protected]:rraileanu/auto-drac.git
cd auto-drac
pip install -r requirements.txt
git clone https://github.com/openai/baselines.git
cd baselines
python setup.py install
pip install procgen
Instructions
cd auto-drac
Train DrAC with crop augmentation on BigFish
python train.py --env_name bigfish --aug_type crop
Train UCB-DrAC on BigFish
python train.py --env_name bigfish --use_ucb
Train RL2-DrAC on BigFish
python train.py --env_name bigfish --use_rl2
Train Meta-DrAC on BigFish
python train.py --env_name bigfish --use_meta
Procgen Results
UCB-DrAC achieves state-of-the-art performance on the Procgen benchmark (easy mode), significantly improving the agent's generalization ability over standard RL methods such as PPO.
Test Results on Procgen
Train Results on Procgen
Agent Videos
You can find some videos of the agent's behavior while training on our website.
Acknowledgements
This code was based on an open sourced PyTorch implementation of PPO.
We also used kornia for some of the augmentations.
38 Dec 12, 2022
111 Dec 8, 2022
3k Dec 31, 2022
997 Dec 30, 2022
3 Jan 23, 2022
8 Sep 30, 2022
45 Dec 8, 2022
1 Dec 30, 2021
29 Dec 7, 2022
364 Dec 28, 2022
38 Oct 14, 2022
41 Oct 21, 2022
25 Apr 28, 2022
756 Dec 27, 2022
399 Jan 2, 2023
18 Dec 17, 2022
5 Jul 18, 2022
1 Dec 29, 2021
5 May 13, 2022