RLStructures is a library to facilitate the implementation of new reinforcement learning algorithms.

Last update: Nov 18, 2022

Related tags

Overview

TL;DR

TL;DR: RLStructures is a lightweight Python library that provides simple APIs as well as data structures that make as few assumptions as possible about the structure of your agent or your task, while allowing for transparently executing multiple policies on multiple environments in parallel (incl. multiple GPUs). It thus facilitates the implementation of RL algorithms while avoiding complex abstractions.

Why/What?

RL research addresses multiple aspects of RL like hierarchical policies, option-based policies, goal-oriented policies, structured input/output spaces, transformer-based policies, etc. while many available tools are specific to particular settings.

We propose RLStructures as a way to i) simulate multiple policies, multiple models and multiple environments simultaneously at scale ii) define complex loss functions and iii) quickly implement various policy architectures.

The main principle of RLStructures is to allow the user to delegate the sampling of trajectories and episodes to the library so they can spend most of their time on the interesting part of RL research: developing new models and algorithms.

RLStructures is easy to use: it has very few simple interfaces that can be learned in one hour by reading the tutorials. It is not a RL Alorithms catalog, but a library to do RL Research. For illustration purposes it comes with multiple RL algorithms including A2C, PPO, DDQN and SAC.

Installation

Install from source by running the following inside the repo:

pip install .

Learning RLStructures

http://facebookresearch.github.io/rlstructures

The complete documentation is available at http://facebookresearch.github.io/rlstructures and tutorials can be found in the tutorial/ directory. The example algorithms are provided in raglos/.

A facebook group is also open for discussion.

List of Papers using rlstructures

Citing RLStructures

Please use this bibtex if you want to cite this repository in your publications:

    @misc{rlstructures,
        author = {L. Denoyer, D. Rothermel and X. Martinet},
        title = {{RLStructures - A simple library for RL research}},
        year = {2021},
        publisher = {GitHub},
        journal = {GitHub repository},
        howpublished = {\url{https://gitHub.com/facebookresearch/rlstructures}},
    }

Author: Ludovic Denoyer
Co-authors: Danielle Rothermel, Xavier Martinet
Other contributors: many....

License

rlstructures is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

pyprobables is a pure-python library for probabilistic data structures

pyprobables is a pure-python library for probabilistic data structures. The goal is to provide the developer with a pure-python implementation of common probabilistic data-structures to use in their work.

86 Dec 25, 2022

Python library for doing things with Grid-like structures

gridthings Python library for doing things with Grid-like structures Development This project uses poetry for dependency management, pre-commit for li

2 Dec 21, 2021

A Python library for electronic structure pre/post-processing

PyProcar PyProcar is a robust, open-source Python library used for pre- and post-processing of the electronic structure data coming from DFT calculati

124 Dec 7, 2022

My notes on Data structure and Algos in golang implementation and python

My notes on DS and Algo Table of Contents Arrays LinkedList Trees Types of trees: Tree/Graph Traversal Algorithms Heap Priorty Queue Trie Graphs Graph

0 Feb 13, 2022

A Python dictionary implementation designed to act as an in-memory cache for FaaS environments

faas-cache-dict A Python dictionary implementation designed to act as an in-memory cache for FaaS environments. Formally you would describe this a mem

3 Dec 13, 2022

A Python implementation of red-black trees

Python red-black trees A Python implementation of red-black trees. This code was originally copied from programiz.com, but I have made a few tweaks to

7 Oct 20, 2022

This repo represents all we learned and are learning in Data Structure course.

DataStructure Journey This repo represents all we learned and are learning in Data Structure course which is based on CLRS book and is being taught by

3 Jan 22, 2022

Reinforcement Learning Coach by Intel AI Lab enables easy experimentation with state of the art Reinforcement Learning algorithms

Coach Coach is a python reinforcement learning framework containing implementation of many state-of-the-art algorithms. It exposes a set of easy-to-us

2.2k Jan 5, 2023

An example project demonstrating how the Autonomous Learning Library can be used to build new reinforcement learning agents.

About This repository shows how Autonomous Learning Library can be used to build new reinforcement learning agents. In particular, it contains a model

5 Aug 30, 2022

Modular Deep Reinforcement Learning framework in PyTorch. Companion library of the book "Foundations of Deep Reinforcement Learning".

SLM Lab Modular Deep Reinforcement Learning framework in PyTorch. Documentation: https://slm-lab.gitbook.io/slm-lab/ BeamRider Breakout KungFuMaster M

1.1k Dec 24, 2022

Pytorch implementation of Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization https://arxiv.org/abs/2008.11646

[TCSVT] Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization LPN [Paper] NEWs Prerequisites Python 3.6 GPU Memory = 8G Numpy 1.

46 Dec 14, 2022

Visual analysis and diagnostic tools to facilitate machine learning model selection.

Yellowbrick Visual analysis and diagnostic tools to facilitate machine learning model selection. What is Yellowbrick? Yellowbrick is a suite of visual

3.9k Dec 30, 2022

Patchwork is a web-based patch tracking system designed to facilitate the contribution and management of contributions to an open-source project.

Patchwork Patchwork is a patch tracking system for community-based projects. It is intended to make the patch management process easier for both the p

220 Nov 29, 2022

All Tools In One is a Script Developed with Python3. It gathers a total of 14 Discord tools (including a RAT, a Raid Tool, a Token Grabber, a Crash Video Maker, etc). It has a pleasant and intuitive interface to facilitate the use of all with help and explanations for each of them.

[Discord] - All Tools In One [Discord] - All Tools In One is a Script Gathering for Windows systems written in Python. Disclaimer This project was cre

484 Jan 1, 2023

ToeholdTools is a Python package and desktop app designed to facilitate analyzing and designing toehold switches, created as part of the 2021 iGEM competition.

ToeholdTools Category Status Repository Package Build Quality A library for the analysis of toehold switch riboregulators created by the iGEM team Cit

0 Dec 1, 2021

A Python package to facilitate research on building and evaluating automated scoring models.

Rater Scoring Modeling Tool Introduction Automated scoring of written and spoken test responses is a growing field in educational natural language pro

59 Oct 10, 2022

WaveFake: A Data Set to Facilitate Audio DeepFake Detection

WaveFake: A Data Set to Facilitate Audio DeepFake Detection This is the code repository for our NeurIPS 2021 (Track on Datasets and Benchmarks) paper

27 Dec 22, 2022

DefAP is a program developed to facilitate the exploration of a material's defect chemistry

DefAP is a program developed to facilitate the exploration of a material's defect chemistry. A large number of features are provided and rapid exploration is supported through the use of autoplotting with carefully considered automatic data labelling and simplification options enabling production of publication quality plots.

6 Oct 25, 2022

Chalice - A tool to facilitate Python based lambda deployment

Chalice is a tool to facilitate Python based lambda deployment. This repo contains the output of my basic exploration of this tool.

1 Feb 3, 2022

Comments

Allow for `pip install .`

Add a setup.py and update the requirements.txt file. Now pip install . will install all necessary requirements. After pip install, ran the following commands successfully from the top level directory:

python tutorial/tutorial_agent.py
python tutorial/tutorial_environments.py
python tutorial/tutorial_multiprocess_trajectory_batcher.py
python tutorial/tutorial_datastructures.py
python tutorial/tutorial_multiprocess_episode_batcher.py
python tutorial/tutorial_a2c_with_infinite_env/main_a2c.py
python tutorial/tutorial_from_reinforce_to_a2c/main_a2c.py
python tutorial/tutorial_recurrent_policy/main_a2c.py
python tutorial/tutorial_reinforce/main_reinforce.py
python tutorial/tutorial_reinforce_with_evaluation/main_reinforce.py

python rlalgos/template_exp.py

python rlalgos/a2c/run_cartpole.py -cd rlalgos/a2c -cn config.yaml
python rlalgos/a2c/run_cartpole_pomdp.py -cd rlalgos/a2c -cn config.yaml
python rlalgos/dqn/run_q_cartpole.py -cd rlalgos/dqn -cn config.yaml
python rlalgos/ppo/run_cartpole.py -cd rlalgos/ppo -cn config.yaml
python rlalgos/ppo/run_cartpole_pomdp.py -cd rlalgos/ppo -cn config.yaml
python rlalgos/sac/run_cartpole.py -cd rlalgos/sac -cn config.yaml

Also added a .gitignore.

CLA Signed

opened by drothermel 0

Adding Code of Conduct file

This is pull request was created automatically because we noticed your project was missing a Code of Conduct file.

Code of Conduct files facilitate respectful and constructive communities by establishing expected behaviors for project contributors.

This PR was crafted with love by Facebook's Open Source Team.
CLA Signed

opened by facebook-github-bot 0
Adding Contributing file

This is pull request was created automatically because we noticed your project was missing a Contributing file.

CONTRIBUTING files explain how a developer can contribute to the project - which you should actively encourage.

This PR was crafted with love by Facebook's Open Source Team.
CLA Signed

opened by facebook-github-bot 0

RLStructures is a library to facilitate the implementation of new reinforcement learning algorithms.

Related tags

Overview

TL;DR

Why/What?

Installation

Learning RLStructures

List of Papers using rlstructures

Citing RLStructures

License

You might also like...

pyprobables is a pure-python library for probabilistic data structures

Python library for doing things with Grid-like structures

A Python library for electronic structure pre/post-processing

My notes on Data structure and Algos in golang implementation and python

A Python dictionary implementation designed to act as an in-memory cache for FaaS environments

A Python implementation of red-black trees

This repo represents all we learned and are learning in Data Structure course.

Reinforcement Learning Coach by Intel AI Lab enables easy experimentation with state of the art Reinforcement Learning algorithms

An example project demonstrating how the Autonomous Learning Library can be used to build new reinforcement learning agents.

Modular Deep Reinforcement Learning framework in PyTorch. Companion library of the book "Foundations of Deep Reinforcement Learning".

Pytorch implementation of Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization https://arxiv.org/abs/2008.11646

Visual analysis and diagnostic tools to facilitate machine learning model selection.

Patchwork is a web-based patch tracking system designed to facilitate the contribution and management of contributions to an open-source project.

All Tools In One is a Script Developed with Python3. It gathers a total of 14 Discord tools (including a RAT, a Raid Tool, a Token Grabber, a Crash Video Maker, etc). It has a pleasant and intuitive interface to facilitate the use of all with help and explanations for each of them.

ToeholdTools is a Python package and desktop app designed to facilitate analyzing and designing toehold switches, created as part of the 2021 iGEM competition.

A Python package to facilitate research on building and evaluating automated scoring models.

WaveFake: A Data Set to Facilitate Audio DeepFake Detection

DefAP is a program developed to facilitate the exploration of a material's defect chemistry

Chalice - A tool to facilitate Python based lambda deployment

Comments

Allow for `pip install .`

Adding Code of Conduct file

Adding Contributing file

Owner

Facebook Research

A collection of data structures and algorithms I'm writing while learning

This repository is a compilation of important Data Structures and Algorithms based on Python.

This repository is for adding codes of data structures and algorithms, leetCode, hackerrank etc solutions in different languages

Leetcode solutions - All algorithms implemented in Python 3 (for education)

One-Stop Destination for codes of all Data Structures & Algorithms

This repo is all about different data structures and algorithms..

This Repository consists of my solutions in Python 3 to various problems in Data Structures and Algorithms

Webtesting for course Data Structures & Algorithms

Final Project for Practical Python Programming and Algorithms for Data Analysis

Basic sort and search algorithms written in python.