Meta-World
Meta-World is an open-source simulated benchmark for meta-reinforcement learning and multi-task learning consisting of 50 distinct robotic manipulation tasks. We aim to provide task distributions that are sufficiently broad to evaluate meta-RL algorithms' generalization ability to new behaviors.
For more background information, please refer to our website and the accompanying conference publication, which provides baseline results for 8 state-of-the-art meta- and multi-task RL algorithms.
Table of Contents
- Installation
- Using the benchmark
- Citing Meta-World
- Accompanying Baselines
- Become a Contributor
- Acknowledgements
Join the Community
Join our mailing list: [email protected] for infrequent announcements about the status of the benchmark, critical bugs and known issues before conference deadlines, and future plans, please
Need some help? Have a question which is not quite a bug and not quite a feature request?
Join the community Slack by filling out this Google Form.
Installation
Meta-World is based on MuJoCo, which has a proprietary dependency we can't set up for you. Please follow the instructions in the mujoco-py package for help. Once you're ready to install everything, run:
pip install git+https://github.com/rlworkgroup/metaworld.git@master#egg=metaworld
Alternatively, you can clone the repository and install an editable version locally:
git clone https://github.com/rlworkgroup/metaworld.git
cd metaworld
pip install -e .
Using the benchmark
Here is a list of benchmark environments for meta-RL (ML*) and multi-task-RL (MT*):
- ML1 is a meta-RL benchmark environment which tests few-shot adaptation to goal variation within single task. You can choose to test variation within any of 50 tasks for this benchmark.
- ML10 is a meta-RL benchmark which tests few-shot adaptation to new tasks. It comprises 10 meta-train tasks, and 3 test tasks.
- ML45 is a meta-RL benchmark which tests few-shot adaptation to new tasks. It comprises 45 meta-train tasks and 5 test tasks.
- MT10, MT1, and MT50 are multi-task-RL benchmark environments for learning a multi-task policy that perform 10, 1, and 50 training tasks respectively. MT1 is similar to ML1 becau you can choose to test variation within any of 50 tasks for this benchmark. In the original Metaworld experiments, we augment MT10 and MT50 environment observations with a one-hot vector which identifies the task. We don't enforce how users utilize task one-hot vectors, however one solution would be to use a Gym wrapper such as this one
Basics
We provide a Benchmark API, that allows constructing environments following the gym.Env interface.
To use a Benchmark, first construct it (this samples the tasks allowed for one run of an algorithm on the benchmark). Then, construct at least one instance of each environment listed in benchmark.train_classes and benchmark.test_classes. For each of those environments, a task must be assigned to it using env.set_task(task) from benchmark.train_tasks and benchmark.test_tasks, respectively. Tasks can only be assigned to environments which have a key in benchmark.train_classes or benchmark.test_classes matching task.env_name.
Please see below for some small examples using this API.
Running ML1 or MT1
import metaworld
import random
print(metaworld.ML1.ENV_NAMES) # Check out the available environments
ml1 = metaworld.ML1('pick-place-v1') # Construct the benchmark, sampling tasks
env = ml1.train_classes['pick-place-v1']() # Create an environment with task `pick_place`
task = random.choice(ml1.train_tasks)
env.set_task(task) # Set task
obs = env.reset() # Reset environment
a = env.action_space.sample() # Sample an action
obs, reward, done, info = env.step(a) # Step the environoment with the sampled random action
MT1 can be run the same way except that it does not contain any test_tasks
Running a benchmark
Create an environment with train tasks (ML10, MT10, ML45, or MT50):
import metaworld
import random
ml10 = metaworld.ML10() # Construct the benchmark, sampling tasks
training_envs = []
for name, env_cls in ml10.train_classes.items():
env = env_cls()
task = random.choice([task for task in ml10.train_tasks
if task.env_name == name])
env.set_task(task)
training_envs.append(env)
for env in training_envs:
obs = env.reset() # Reset environment
a = env.action_space.sample() # Sample an action
obs, reward, done, info = env.step(a) # Step the environoment with the sampled random action
Create an environment with test tasks (this only works for ML10 and ML45, since MT10 and MT50 don't have a separate set of test tasks):
import metaworld
import random
ml10 = metaworld.ML10() # Construct the benchmark, sampling tasks
testing_envs = []
for name, env_cls in ml10.test_classes.items():
env = env_cls()
task = random.choice([task for task in ml10.test_tasks
if task.env_name == name])
env.set_task(task)
testing_envs.append(env)
for env in testing_envs:
obs = env.reset() # Reset environment
a = env.action_space.sample() # Sample an action
obs, reward, done, info = env.step(a) # Step the environoment with the sampled random action
Citing Meta-World
If you use Meta-World for academic research, please kindly cite our CoRL 2019 paper the using following BibTeX entry.
@inproceedings{yu2019meta,
title={Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning},
author={Tianhe Yu and Deirdre Quillen and Zhanpeng He and Ryan Julian and Karol Hausman and Chelsea Finn and Sergey Levine},
booktitle={Conference on Robot Learning (CoRL)},
year={2019}
eprint={1910.10897},
archivePrefix={arXiv},
primaryClass={cs.LG}
url={https://arxiv.org/abs/1910.10897}
}
Accompanying Baselines
If you're looking for implementations of the baselines algorithms used in the Metaworld conference publication, please look at our sister directory, Garage. Note that these aren't the exact same baselines that were used in the original conference publication, however they are true to the original baselines.
Become a Contributor
We welcome all contributions to Meta-World. Please refer to the contributor's guide for how to prepare your contributions.
Acknowledgements
Meta-World is a work by Tianhe Yu (Stanford University), Deirdre Quillen (UC Berkeley), Zhanpeng He (Columbia University), Ryan Julian (University of Southern California), Karol Hausman (Google AI), Chelsea Finn (Stanford University) and Sergey Levine (UC Berkeley).
The code for Meta-World was originally based on multiworld, which is developed by Vitchyr H. Pong, Murtaza Dalal, Ashvin Nair, Shikhar Bahl, Steven Lin, Soroush Nasiriany, Kristian Hartikainen and Coline Devin. The Meta-World authors are grateful for their efforts on providing such a great framework as a foundation of our work. We also would like to thank Russell Mendonca for his work on reward functions for some of the environments.
![[BUG] Fix reset() method for v2](https://avatars.githubusercontent.com/u/31506965?v=4)
![[BUG] ValueError during training procedure](https://avatars.githubusercontent.com/u/51039267?v=4)
2.2k Jan 5, 2023
404 Dec 25, 2022
213 Jan 5, 2023
2.4k Dec 29, 2022
29.6k Jan 1, 2023
1.5k Dec 30, 2022
1.6k Jan 9, 2023
3.7k Jan 1, 2023
3.3k Jan 5, 2023
3.2k Jan 2, 2023
3.1k Dec 29, 2022
10k Jan 7, 2023
5.4k Jan 4, 2023
1.1k Dec 26, 2022
117 Dec 12, 2022
653 Jan 6, 2023
346 Jan 3, 2023
123 Dec 23, 2022
