CoRe: Contrastive Recurrent State-Space Models

Overview

CoRe: Contrastive Recurrent State-Space Models

This code implements the CoRe model and reproduces experimental results found in
Robust Robotic Control from Pixels using Contrastive Recurrent State-Space models
NeurIPS Deep Reinforcement Learning Workshop 2021
Nitish Srivastava, Walter Talbott, Martin Bertran Lopez, Shuangfei Zhai & Joshua M. Susskind
[paper]

cartpole

cheetah

walker

Requirements and Installation

Clone this repository and then execute the following steps. See setup.sh for an example of how to run these steps on a Ubuntu 18.04 machine.

  • Install dependencies.

    apt install -y libgl1-mesa-dev libgl1-mesa-glx libglew-dev \
            libosmesa6-dev software-properties-common net-tools unzip \
            virtualenv wget xpra xserver-xorg-dev libglfw3-dev patchelf xvfb ffmpeg
    
  • Download the DAVIS 2017 dataset. Make sure to select the 2017 TrainVal - Images and Annotations (480p). The training images will be used as distracting backgrounds. The DAVIS directory should be in the same directory as the code. Check that ls ./DAVIS/JPEGImages/480p/... shows 90 video directories.

  • Install MuJoCo 2.1.

    • Download MuJoCo version 2.1 binaries for Linux or macOS.
    • Unzip the downloaded mujoco210 directory into ~/.mujoco/mujoco210.
  • Install MuJoCo 2.0 (For robosuite experiments only).

    • Download MuJoCo version 2.0 binaries for Linux or macOS.
    • Unzip the downloaded directory and move it into ~/.mujoco/.
    • Symlink mujoco200_linux (or mujoco200_macos) to mujoco200.
    ln -s ~/.mujoco/mujoco200_linux ~/.mujoco/mujoco200
    
    • Place the license key at ~/.mujoco/mjkey.txt.
    • Add the MuJoCo binaries to LD_LIBRARY_PATH.
    export LD_LIBRARY_PATH=$HOME/.mujoco/mujoco200/bin:$LD_LIBRARY_PATH
    
  • Setup EGL GPU rendering (if a GPU is available).

    • To ensure that the GPU is prioritized over the CPU for EGL rendering
    cp 10_nvidia.json /usr/share/glvnd/egl_vendor.d/
    
    • Create a dummy nvidia directory so that mujoco_py builds the extensions needed for GPU rendering.
    mkdir -p /usr/lib/nvidia-000
    export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia-000
    
  • Create a conda environment.

    For Distracting Control Suite

    conda env create -f conda_env.yml
    

    For Robosuite

    conda env create -f conda_env_robosuite.yml
    

Training

  • The CoRe model can be trained on the Distracting Control Suite as follows:

    conda activate core
    MUJOCO_GL=egl CUDA_VISIBLE_DEVICES=0 python train.py --config configs/dcs/core.yaml 
    

The training artifacts, including tensorboard logs and videos of validation rollouts will be written in ./artifacts/.

To change the distraction setting, modify the difficulty parameter in configs/dcs/core.yaml. Possible values are ['easy', 'medium', 'hard', 'none', 'hard_bg'].

To change the domain, modify the domain parameter in configs/dcs/core.yaml. Possible values are ['ball_in_cup', 'cartpole', 'cheetah', 'finger', 'reacher', 'walker'].

  • To train on Robosuite (Door Task, Franka Panda Arm)

    • Using RGB image and proprioceptive inputs.
    conda activate core_robosuite
    MUJOCO_GL=egl CUDA_VISIBLE_DEVICES=0 python train.py --config configs/robosuite/core.yaml
    
    • Using RGB image inputs only.
    conda activate core_robosuite
    MUJOCO_GL=egl CUDA_VISIBLE_DEVICES=0 python train.py --config configs/robosuite/core_imageonly.yaml
    

Citation

@article{srivastava2021core,
    title={Robust Robotic Control from Pixels using Contrastive Recurrent State-Space Models}, 
    author={Nitish Srivastava and Walter Talbott and Martin Bertran Lopez and Shuangfei Zhai and Josh Susskind},
    journal={NeurIPS Deep Reinforcement Learning Workshop},
    year={2021}
}

License

This code is released under the LICENSE terms.

You might also like...
LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models
LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models

LaneDet is an open source lane detection toolbox based on PyTorch that aims to pull together a wide variety of state-of-the-art lane detection models. Developers can reproduce these SOTA methods and build their own methods.

Library for implementing reservoir computing models (echo state networks) for multivariate time series classification and clustering.
Library for implementing reservoir computing models (echo state networks) for multivariate time series classification and clustering.

Framework overview This library allows to quickly implement different architectures based on Reservoir Computing (the family of approaches popularized

PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+
PaddleViT: State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 2.0+

PaddlePaddle Vision Transformers State-of-the-art Visual Transformer and MLP Models for PaddlePaddle 🤖 PaddlePaddle Visual Transformers (PaddleViT or

LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.
LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.

LWCC: A LightWeight Crowd Counting library for Python LWCC is a lightweight crowd counting framework for Python. It wraps four state-of-the-art models

PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models.
PySlowFast: video understanding codebase from FAIR for reproducing state-of-the-art video models.

PySlowFast PySlowFast is an open source video understanding codebase from FAIR that provides state-of-the-art video classification models with efficie

TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

TorchMultimodal (Alpha Release) Introduction TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayesian-Torch is designed to be flexible and seamless in extending a deterministic deep neural network architecture to corresponding Bayesian form by simply replacing the deterministic layers with Bayesian layers.

The coda and data for
The coda and data for "Measuring Fine-Grained Domain Relevance of Terms: A Hierarchical Core-Fringe Approach" (ACL '21)

We propose a hierarchical core-fringe learning framework to measure fine-grained domain relevance of terms – the degree that a term is relevant to a broad (e.g., computer science) or narrow (e.g., deep learning) domain.

Power Core Simulator!

Power Core Simulator Power Core Simulator is a simulator based off the Roblox game "Pinewood Builders Computer Core". In this simulator, you can choos

Comments
  • Correct Hyperparameter Configuration

    Correct Hyperparameter Configuration

    Hey, I am currently working on reproducing your work and evaluating it on different environments. Unfortunately, the committed hyperparameters seem to differ from the mentioned hyperparameters in the paper.

    Especially, the rendering size and crop size appears to be unrealistic in the code (256, and 84 crop). Another missing information is the decoder, which could be useful for more exact representation of your plots.

    Could you share which configuration you used exactly to achieve the results on DMC?

    Thank you very much in advance!

    opened by sebimarkgraf 0
  • Releasing results datasets

    Releasing results datasets

    Hi,

    CoRE achieves very impressive results in the Distracting Control Suite. Would it be possible for you to release a dataset of the results that are reported in the paper (i.e., not just the numbers in the tables, but the data used to produce plots such as Figure 8 in Appendix B)?

    opened by trevormcinroe 0
Owner
Apple
Apple
[SIGIR22] Official PyTorch implementation for "CORE: Simple and Effective Session-based Recommendation within Consistent Representation Space".

CORE This is the official PyTorch implementation for the paper: Yupeng Hou, Binbin Hu, Zhiqiang Zhang, Wayne Xin Zhao. CORE: Simple and Effective Sess

RUCAIBox 26 Dec 19, 2022
Space Time Recurrent Memory Network - Pytorch

Space Time Recurrent Memory Network - Pytorch (wip) Implementation of Space Time Recurrent Memory Network, recurrent network competitive with attentio

Phil Wang 50 Nov 7, 2021
Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Noise Contrastive Estimation for pyTorch Overview This repository contains a re-implementation of the Noise Contrastive Estimation algorithm, implemen

Denis Emelin 42 Nov 24, 2022
FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

FuseDream This repo contains code for our paper (paper link): FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizat

XCL 191 Dec 31, 2022
Space robot - (Course Project) Using the space robot to capture the target satellite that is disabled and spinning, then stabilize and fix it up

Space robot - (Course Project) Using the space robot to capture the target satellite that is disabled and spinning, then stabilize and fix it up

Mingrui Yu 3 Jan 7, 2022
Quickly comparing your image classification models with the state-of-the-art models (such as DenseNet, ResNet, ...)

Image Classification Project Killer in PyTorch This repo is designed for those who want to start their experiments two days before the deadline and ki

null 349 Dec 8, 2022
Saeed Lotfi 28 Dec 12, 2022
Pytorch implementation of "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling"

RNN-for-Joint-NLU Pytorch implementation of "Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling"

Kim SungDong 194 Dec 28, 2022
PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models Code accompanying CVPR'20 paper of the same title. Paper lin

Alex Damian 7k Dec 30, 2022
QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art Machine Learning models.

null 152 Jan 2, 2023