Discovering and Achieving Goals via World Models

Related tags

Deep Learning lexa
Overview

Discovering and Achieving Goals via World Models

[Project Website] [Benchmark Code] [Video (2min)] [Oral Talk (13min)] [Paper]

Russell Mendonca*1, Oleh Rybkin*2, Kostas Daniilidis2, Danijar Hafner3,4, Deepak Pathak1
(* equal contribution, random order)

1Carnegie Mellon University
2University of Pennsylvania
3Google Research, Brain Team
4University of Toronto

Official implementation of the Lexa agent from the paper Discovering and Achieving Goals via World Models.

Setup

Create the conda environment by running :

conda env create -f environment.yml

Clone the lexa-benchmark repo, and modify the python path
export PYTHONPATH= /lexa:

Export the following variables for rendering
export MUJOCO_RENDERER=egl; export MUJOCO_GL=egl

Training

First source the environment : source activate lexa

For training, run :

export CUDA_VISIBLE_DEVICES=
   
      
python train.py --configs defaults 
    
      --task 
     
       --logdir

where method can be lexa_temporal, lexa_cosine, ddl, diayn or gcsl
Supported tasks are dmc_walker_walk, dmc_quadruped_run, robobin, kitchen, joint

To view the graphs and gifs during training, run tensorboard --logdir

Bibtex

If you find this code useful, please cite:

@misc{lexa2021,
    title={Discovering and Achieving Goals via World Models},
    author={Mendonca, Russell and Rybkin, Oleh and
    Daniilidis, Kostas and Hafner, Danijar and Pathak, Deepak},
    year={2021},
    Booktitle={NeurIPS}
}

Acknowledgements

This code was developed using Dreamer V2 and Plan2Explore.

You might also like...
Ever felt tired after preprocessing the dataset, and not wanting to write any code further to train your model? Ever encountered a situation where you wanted to record the hyperparameters of the trained model and able to retrieve it afterward? Models Playground is here to help you do that. Models playground allows you to train your models right from the browser.
Ever felt tired after preprocessing the dataset, and not wanting to write any code further to train your model? Ever encountered a situation where you wanted to record the hyperparameters of the trained model and able to retrieve it afterward? Models Playground is here to help you do that. Models playground allows you to train your models right from the browser.

Models Playground 🗂️ Upload a Preprocessed Dataset 🌠 Choose whether to perform Classification or Regression 🦹 Enter the Dependent Variable ?

null 19 Dec 10, 2022
PyTorch CZSL framework containing GQA, the open-world setting, and the CGE and CompCos methods.
PyTorch CZSL framework containing GQA, the open-world setting, and the CGE and CompCos methods.

Compositional Zero-Shot Learning This is the official PyTorch code of the CVPR 2021 works Learning Graph Embeddings for Compositional Zero-shot Learni

EML Tübingen 70 Dec 27, 2022
A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.
A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

ManhattanSLAM Authors: Raza Yunus, Yanyan Li and Federico Tombari ManhattanSLAM is a real-time SLAM library for RGB-D cameras that computes the camera

null 117 Dec 28, 2022
Source code and dataset for ACL2021 paper: "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning".

ERICA Source code and dataset for ACL2021 paper: "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive L

THUNLP 75 Nov 2, 2022
The world's simplest facial recognition api for Python and the command line
The world's simplest facial recognition api for Python and the command line

Face Recognition You can also read a translated version of this file in Chinese 简体中文版 or in Korean 한국어 or in Japanese 日本語. Recognize and manipulate fa

Adam Geitgey 46.9k Jan 3, 2023
A toolkit for making real world machine learning and data analysis applications in C++

dlib C++ library Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real worl

Davis E. King 11.6k Jan 1, 2023
RL and distillation in CARLA using a factorized world model
RL and distillation in CARLA using a factorized world model

World on Rails Learning to drive from a world on rails Dian Chen, Vladlen Koltun, Philipp Krähenbühl, arXiv techical report (arXiv 2105.00636) This re

Dian Chen 131 Dec 16, 2022
HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset (ICCV 2021)
HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset (ICCV 2021)

Code for HDR Video Reconstruction HDR Video Reconstruction: A Coarse-to-fine Network and A Real-world Benchmark Dataset (ICCV 2021) Guanying Chen, Cha

Guanying Chen 64 Nov 19, 2022
Repo for FUZE project. I will also publish some Linux kernel LPE exploits for various real world kernel vulnerabilities here. the samples are uploaded for education purposes for red and blue teams.

Linux_kernel_exploits Some Linux kernel exploits for various real world kernel vulnerabilities here. More exploits are yet to come. This repo contains

Wei Wu 472 Dec 21, 2022
Comments
  • Issue in enviornment setup

    Issue in enviornment setup

    I followed the setup instructions in the README and when I tried to run the train script with: python train.py --configs defaults lexa_temporal --task dmc_walker_walk --logdir ~/logdir/lexa_temporal_walker

    I got the following error:

    Traceback (most recent call last):
  File "train.py", line 9, in <module>
    from dreamer import Dreamer, setup_dreamer, create_envs, count_steps, make_dataset, parse_dreamer_args
  File "/home/ei-lab/rbhagat8/lexa/lexa/dreamer.py", line 27, in <module>
    import wrappers
  File "/home/ei-lab/rbhagat8/lexa/lexa/wrappers.py", line 5, in <module>
    import mujoco_py
  File "/home/ei-lab/miniconda3/envs/lexa/lib/python3.6/site-packages/mujoco_py/__init__.py", line 3, in <module>
    from mujoco_py.builder import cymj, ignore_mujoco_warnings, functions, MujocoException
  File "/home/ei-lab/miniconda3/envs/lexa/lib/python3.6/site-packages/mujoco_py/builder.py", line 510, in <module>
    cymj = load_cython_ext(mujoco_path)
  File "/home/ei-lab/miniconda3/envs/lexa/lib/python3.6/site-packages/mujoco_py/builder.py", line 106, in load_cython_ext
    mod = load_dynamic_ext('cymj', cext_so_path)
  File "/home/ei-lab/miniconda3/envs/lexa/lib/python3.6/site-packages/mujoco_py/builder.py", line 125, in load_dynamic_ext
    return loader.load_module()
ImportError: /home/ei-lab/miniconda3/envs/lexa/bin/../lib/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /lib/x86_64-linux-gnu/libOSMesa.so.8)

    But I was able to fix this using these two commands: rm ~/miniconda3/envs/lexa/bin/../lib/libstdc++.so.6 cp /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ~/miniconda3/envs/lexa/bin/../lib/

    But after that, when I ran the train script again using the same command as above I got this error:

    Traceback (most recent call last):
  File "train.py", line 9, in <module>
    from dreamer import Dreamer, setup_dreamer, create_envs, count_steps, make_dataset, parse_dreamer_args
  File "/home/ei-lab/rbhagat8/lexa/lexa/dreamer.py", line 29, in <module>
    import envs
  File "/home/ei-lab/rbhagat8/lexa-benchmark/envs/__init__.py", line 1, in <module>
    from envs.robobin import RoboBinEnv
  File "/home/ei-lab/rbhagat8/lexa-benchmark/envs/robobin.py", line 8, in <module>
    import metaworld.envs.mujoco.sawyer_xyz.v1 as sawyer
  File "/home/ei-lab/rbhagat8/lexa-benchmark/metaworld/__init__.py", line 7, in <module>
    import metaworld.envs.mujoco.env_dict as _env_dict
ModuleNotFoundError: No module named 'metaworld.envs.mujoco'

    I tried reinstalling metaworld with pip install metaworld but it still results in the same error. Any advice?

    opened by rishavb123 7
  • Why do train episodes contain eval goals?

    Why do train episodes contain eval goals?

    When training a model with lexa_cosine on dmc_walker_walk and visualising the episodes contained in {logdir}\train_eps, I noticed that these episodes had for image_goal an image from the eval set of goals. Why ? And how to visualise the actual train goals ?

    opened by LinaMezghani 4
  • Issue at training: Tensor had Inf values

    Issue at training: Tensor had Inf values

    Hi! I'm trying to run LEXA, but I ran into the following issues:

    1. ensemble_loss: Tensor had Inf values. When I ran python train.py --configs defaults lexa_temporal --task kitchen --logdir logdir, it trained for 1 step, and then after ~10 minutes (without any other ouput) it reported back the following error: Screen Shot 2022-02-15 at 9 48 04 PM

    Same for lexa_cosine and diayn on kitchen.

    1. Shape mismatch. It happened for lexa_temporal on dmc_walker_walk and dmc_quadruped_run. Screen Shot 2022-02-15 at 9 54 38 PM Screen Shot 2022-02-15 at 9 55 11 PM

    2. Param shape mismatch. Only happened for python train.py --configs defaults ddl --task kitchen --logdir logdir Screen Shot 2022-02-15 at 9 56 41 PM

    My setup generally follows the instruction in README. However, since I'm using a new version of mujoco (mujoco-2.1.1), I had to use python==3.7, mujoco-py==2.1.2.14, and dm-control==0.0.425341097. The rest of packages (including tensorflow==2.4.0) are kept the same.

    This is a complete list of my packages: Screen Shot 2022-02-15 at 9 58 53 PM

    Screen Shot 2022-02-15 at 9 59 55 PM

    Other specifications: Ubuntu 18.04 CUDA 11.3

    Thanks in advance!

    opened by lucys0 2
Owner
Oleg Rybkin
Ph.D. student with Kostas Daniilidis. I work on making machines think about the future.
Oleg Rybkin
GitHub
Optimizing DR with hard negatives and achieving SOTA first-stage retrieval performance on TREC DL Track (SIGIR 2021 Full Paper).

Optimizing Dense Retrieval Model Training with Hard Negatives Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, Shaoping Ma This repo provi

Jingtao Zhan 99 Dec 27, 2022
Code for the paper Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration

IMAGINE: Language as a Cognitive Tool to Imagine Goals in Curiosity Driven Exploration This repo contains the code base of the paper Language as a Cog

Flowers Team 26 Dec 22, 2022
discovering subdomains, hidden paths, extracting unique links

python-website-crawler discovering subdomains, hidden paths, extracting unique links pip install -r requirements.txt discover subdomain: You can give

merve 4 Sep 5, 2022
Deep learning-based approach to discovering Granger causality networks in multivariate time series

Granger causality discovery for neural networks.

Ian Covert 130 Jan 1, 2023
Supplementary code for SIGGRAPH 2021 paper: Discovering Diverse Athletic Jumping Strategies

SIGGRAPH 2021: Discovering Diverse Athletic Jumping Strategies project page paper demo video Prerequisites Important Notes We suspect there are bugs i

null 54 Dec 6, 2022
Discovering Interpretable GAN Controls [NeurIPS 2020]

GANSpace: Discovering Interpretable GAN Controls Figure 1: Sequences of image edits performed using control discovered with our method, applied to thr

Erik Härkönen 1.7k Jan 3, 2023
DIR-GNN - Discovering Invariant Rationales for Graph Neural Networks

DIR-GNN "Discovering Invariant Rationales for Graph Neural Networks" (ICLR 2022)

Ying-Xin (Shirley) Wu 70 Nov 13, 2022
[NeurIPS 2021] “Improving Contrastive Learning on Imbalanced Data via Open-World Sampling”,

Improving Contrastive Learning on Imbalanced Data via Open-World Sampling Introduction Contrastive learning approaches have achieved great success in

VITA 24 Dec 17, 2022
Learning Generative Models of Textured 3D Meshes from Real-World Images, ICCV 2021

Learning Generative Models of Textured 3D Meshes from Real-World Images This is the reference implementation of "Learning Generative Models of Texture

Dario Pavllo 115 Jan 7, 2023
World Models with TensorFlow 2

World Models This repo reproduces the original implementation of World Models. This implementation uses TensorFlow 2.2. Docker The easiest way to hand

Zac Wellmer 234 Nov 30, 2022