CURL: Contrastive Unsupervised Representations for Reinforcement Learning

Related tags

Deep Learning curl_rainbow
Overview

CURL Rainbow

MIT License

Status: Archive (code is provided as-is, no updates expected)

This is an implementation of CURL: Contrastive Unsupervised Representations for Reinforcement Learning coupled with the Data Efficient Rainbow method for Atari games. The code by default uses the 100k timesteps benchmark and has not been tested for any other setting.

Run the following command (or bash run_curl.sh) with the game as an argument:

python3 main.py --game ms_pacman

To install all dependencies, run bash install.sh.

Comments
  • illegal memory access

    illegal memory access

    When I run the code on TITAN RTX with python3.6, CUDA 10.0.130, pytorch 1.4.0, I got the error like this: File "curl_rainbow/agent.py", line 74, in act return (a * self.support).sum(2).argmax(1).item() RuntimeError: CUDA error: an illegal memory access was encountered

    But I can run this code on cpu. Does anyone know what happens here?

    opened by Rivendile 3
  • T-max problem

    T-max problem

    Why did you choose T-max=100k in your experiment? I think you should train until its convergence. I run your code with T-max=800k on game pong，The CURL and rainbow have the same sample efficiency. Even when T-max=100k, I failed to reproduce the experimental results in the paper.

    opened by zhanghongjie101 3
  • About the evaluation choice

    About the evaluation choice

    Hi Aravind,

    I like your work very much! But I just have one question about the evaluation, when I directly run your code in the game battle zone, sometimes the best performance is not in the end but in the middle, and the performance in the end could be worse and couldn't reach the scores mentioned in the paper. Did you use the best score ever, or only the score in the end in the evaluation? Thank you very much!

    opened by Cohencohenchen 1
  • can't reproduce on pong and hero

    can't reproduce on pong and hero

    I tried the code with default parameter you set.

    But I can't reproduce the reward on pong and hero as reported in the paper. To be precise, the reward is ~-20 vs -16.5 on pong and ~3000 vs ~6000 on hero.

    1604491601803

    1604491647592

    Do you have any idea why this happened? I simply used the command:

    python main.py --game pong/hero

    opened by noahcao 1
  • About Frame skip

    About Frame skip

    In this CURL original paper, it says atari benchmark use 4 frameskip.

    But, in this code, default code has 0 frameskip.

    Is there something point that I miss?

    opened by binmom 1
  • Where does the Query interact with the Rainbow DQN

    Where does the Query interact with the Rainbow DQN

    Based on my understanding of the paper, I believe that the query-keys go into the contrastive learning objective function, while the queries go into the RL algorithm as observations. However, I am unable to find (in the code) where you send the query into the Rainbow DQN as state/observation. Can you please help me with this?

    opened by Deepakgthomas 0
Owner
Aravind Srinivas
Aravind Srinivas
GitHub
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
Code for the paper "Unsupervised Contrastive Learning of Sound Event Representations", ICASSP 2021.

Unsupervised Contrastive Learning of Sound Event Representations This repository contains the code for the following paper. If you use this code or pa

Eduardo Fonseca 81 Dec 22, 2022
Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [2021]

Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations This repo contains the Pytorch implementation of our paper: Revisit

Wouter Van Gansbeke 80 Nov 20, 2022
CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning

CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning This repository contains the code and relevant instructions

XiaoMing 5 Aug 19, 2022
[ICCV'21] Official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations

CrowdNav with Social-NCE This is an official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations by

VITA lab at EPFL 125 Dec 23, 2022
PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

Thalles Silva 1.7k Dec 28, 2022
Supervised Contrastive Learning for Downstream Optimized Sequence Representations

SupCL-Seq ?? Supervised Contrastive Learning for Downstream Optimized Sequence representations (SupCS-Seq) accepted to be published in EMNLP 2021, ext

Hooman Sedghamiz 18 Oct 21, 2022
Proto-RL: Reinforcement Learning with Prototypical Representations

Proto-RL: Reinforcement Learning with Prototypical Representations This is a PyTorch implementation of Proto-RL from Reinforcement Learning with Proto

Denis Yarats 74 Dec 6, 2022
Pretraining Representations For Data-Efficient Reinforcement Learning

Pretraining Representations For Data-Efficient Reinforcement Learning Max Schwarzer, Nitarshan Rajkumar, Michael Noukhovitch, Ankesh Anand, Laurent Ch

Mila 40 Dec 11, 2022
Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX

CQL-JAX This repository implements Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX (FLAX). Implementation is built on

Karush Suri 8 Nov 7, 2022
Reinforcement-learning - Repository of the class assignment questions for the course on reinforcement learning

DSE 314/614: Reinforcement Learning This repository containing reinforcement lea

Manav Mishra 4 Apr 15, 2022
SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING - The Facebook paper about fine tuning RoBERTa with contrastive loss

"# SUPERVISED-CONTRASTIVE-LEARNING-FOR-PRE-TRAINED-LANGUAGE-MODEL-FINE-TUNING" i

Saeed Lotfi 28 Dec 12, 2022
Code for Dual Contrastive Learning for Unsupervised Image-to-Image Translation, NTIRE, CVPRW 2021.

arXiv Dual Contrastive Learning Adversarial Generative Networks (DCLGAN) We provide our PyTorch implementation of DCLGAN, which is a simple yet powerf

null 119 Dec 4, 2022
We evaluate our method on different datasets (including ShapeNet, CUB-200-2011, and Pascal3D+) and achieve state-of-the-art results, outperforming all the other supervised and unsupervised methods and 3D representations, all in terms of performance, accuracy, and training time.

An Effective Loss Function for Generating 3D Models from Single 2D Image without Rendering Papers with code | Paper Nikola Zubić Pietro Lio University

Nikola Zubić 213 Dec 27, 2022
The Unsupervised Reinforcement Learning Benchmark (URLB)

The Unsupervised Reinforcement Learning Benchmark (URLB) URLB provides a set of leading algorithms for unsupervised reinforcement learning where agent

null 259 Dec 26, 2022
Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Noise Contrastive Estimation for pyTorch Overview This repository contains a re-implementation of the Noise Contrastive Estimation algorithm, implemen

Denis Emelin 42 Nov 24, 2022
A resource for learning about deep learning techniques from regression to LSTM and Reinforcement Learning using financial data and the fitness functions of algorithmic trading

A tour through tensorflow with financial data I present several models ranging in complexity from simple regression to LSTM and policy networks. The s

null 195 Dec 7, 2022
Dense Contrastive Learning (DenseCL) for self-supervised representation learning, CVPR 2021.

Dense Contrastive Learning for Self-Supervised Visual Pre-Training This project hosts the code for implementing the DenseCL algorithm for se

Xinlong Wang 491 Jan 3, 2023
Viewmaker Networks: Learning Views for Unsupervised Representation Learning

Viewmaker Networks: Learning Views for Unsupervised Representation Learning Alex Tamkin, Mike Wu, and Noah Goodman Paper link: https://arxiv.org/abs/2

Alex Tamkin 31 Dec 1, 2022