Yet Another Robotics and Reinforcement (YARR) learning framework for PyTorch.

Stephen James

Last update: Dec 27, 2022

Related tags

Deep Learning YARR

Overview

Note: Pirate qualification not needed to use this library.

YARR is Yet Another Robotics and Reinforcement learning framework for PyTorch.

The framework allows for asynchronous training (i.e. agent and learner running in separate processes), which makes it suitable for robot learning. For an example of how to use this framework, see my Attention-driven Robot Manipulation (ARM) repo.

This project is mostly intended for my personal use and facilitate my research.

Install

Ensure you have PyTorch installed. Then simply run:

pip install .

Comments

Question about EnvRunner: multiple environments but one buffer

Hi, Thanks a ton for sharing your amazing framework.

I have one question regarding the EnvRunner class. I apologize if I misunderstood something and the question is not relevant.

The EnvRunner class allows multiple processes to collect samples in separate training environments, but they all seem to add their samples to the same replay buffer. Doesn't this cause a problem when we sample from this buffer (contiguous samples in the buffer are no longer sequentially tied)?

Thanks!

opened by telejesus2 1

Suppress warning when initialize tensor from list of ndarray

<stdin>:1: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

opened by wkentaro 0

Install natsort

To avoid the following error:

Traceback (most recent call last):
  File "./train.py", line 15, in <module>
    from yarr.replay_buffer.prioritized_replay_buffer import (
  File "/home/wkentaro/projectX/.anaconda3/lib/python3.7/site-packages/yarr/replay_buffer/prioritized_replay_buffer.py", line 10, in <module>
    from .uniform_replay_buffer import *
  File "/home/wkentaro/projectX/.anaconda3/lib/python3.7/site-packages/yarr/replay_buffer/uniform_replay_buffer.py", line 22, in <module>
    from natsort import natsort
ModuleNotFoundError: No module named 'natsort'

opened by wkentaro 0

Pass transition.info from env.step to ReplayTransition

Same as multi_task_rollout_generator

https://github.com/stepjam/YARR/blob/ff2128efc7172166e9985cca1310725ff9384d29/yarr/utils/multi_task_rollout_generator.py#L53-L59

opened by wkentaro 0
Compatibility with a debugger

Hi!

Thanks for this package. It is remarkably well written!

However, it seems that yarr is necessarily creating new processes, while a debugger such as pdb does not work at all with sub processes.

What I'd like to do basically is being able to go step by step over my RL algo. Is there a way to do that?

Thanks, PL

opened by guhur 0

Yet Another Robotics and Reinforcement (YARR) learning framework for PyTorch.

Related tags

Overview

Install

Comments

Question about EnvRunner: multiple environments but one buffer

Suppress warning when initialize tensor from list of ndarray

Install natsort

Pass transition.info from env.step to ReplayTransition

Compatibility with a debugger

Owner

Stephen James

Yet Another Reinforcement Learning Tutorial

An image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testingAn image base contains 490 images for learning (400 cars and 90 boats), and another 21 images for testing

Yet another video caption

Bonnet: An Open-Source Training and Deployment Framework for Semantic Segmentation in Robotics.

Pytorch code for ICRA'21 paper: "Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation"

Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX

Reinforcement-learning - Repository of the class assignment questions for the course on reinforcement learning

Libraries, tools and tasks created and used at DeepMind Robotics.

Robotics with GPU computing

The Generic Manipulation Driver Package - Implements a ROS Interface over the robotics toolbox for Python

Reinforcement learning framework and algorithms implemented in PyTorch.

Reinforcement learning library(framework) designed for PyTorch, implements DQN, DDPG, A2C, PPO, SAC, MADDPG, A3C, APEX, IMPALA ...

Learning to trade under the reinforcement learning framework

Another pytorch implementation of FCN (Fully Convolutional Networks)

Another pytorch implementation of FCN (Fully Convolutional Networks)

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

A clear, concise, simple yet powerful and efficient API for deep learning.