This is the pytorch code for the paper Curious Representation Learning for Embodied Intelligence.

Last update: Oct 19, 2022

Related tags

Deep Learning crl

Overview

Curious Representation Learning for Embodied Intelligence

This is the pytorch code for the paper Curious Representation Learning for Embodied Intelligence. This codebase is based on the codebase from Habitat-lab, please see HABITAT_README.md for installation instructions for the repository.

Interactive Pretraining of Embodied Agents

To pretrain agent weights on Matterport3D, please use the following command:

python habitat_baselines/run.py --run-type=train --exp-config habitat_baselines/cvpr_config/pretrain/curiosity_pointnav_pretrain.yaml

The other configs used in the paper may also be found in habitat_baselines/cvpr_config/pretrain.

Downstream ImageNav Pretraining

To finetune weights on ImageNav, please use the following command:

python habitat_baselines/run.py --run-type=train --exp-config habitat_baselines/cvpr_config/imagenav/curiosity_pointnav_gibson_imagenav.yaml

Downstream ObjectNav Pretraining

To finetune weights on ObjectNav, please use the following command:

python habitat_baselines/run.py --run-type=train --exp-config habitat_baselines/cvpr_config/objectnav/curiosity_pointnav_mp3d_objectnav.yaml

Pretrained Weights

The pretrained CRL model from the Matterport3D environment can be downloaded from here

Citing Our Paper

If you find our code useful for your research, please consider citing the following paper, as well as papers included in HABITAT_README.md.

@article{du2021curious,
    author = {Du, Yilun and Gan, Chuang and
    Isola, Phillip},
    title = {Curious Representation Learning
    for Embodied Intelligence},
    journal = {arXiv preprint arXiv:2105.01060},
    year = {2021}
}

Comments

ObjNav MP3D-Val Results

Hi There,

Just a quick question, we would really like to compare with your method on the ObjNav MP3D-Val split in our work. Could we get the best SPL, Soft SPL, Success, and Goal Distance of CRL on this data split?

Many Thanks!

opened by YicongHong 3
Code to compute area of coverage from different RL agent

Thanks for releasing code and pretrained models from this wonderful project! We really enjoy reading and learning from it.

I wonder if it's possible to release code implementation to compute number of areas explored by agents during RL training stage, one for each environment vectorized in paralleled training. Basically the code to compute curve of Figure 3 in the paper on arxiv: I looked through current public code base and didn't find them. If there's anything that I miss please do let me know.

Or I'd also appreciate it if you could point out some utils/functions from Habitat API that you use for computing values of this figure. Thank you so much for help!

opened by xinranliang 0

A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

Exploring simple siamese representation learning This is a PyTorch re-implementation of the SimSiam paper on ImageNet dataset. The results match that

72 Nov 9, 2022

This is the official pytorch implementation for the paper: Instance Similarity Learning for Unsupervised Feature Representation.

ISL This is the official pytorch implementation for the paper: Instance Similarity Learning for Unsupervised Feature Representation, which is accepted

19 May 4, 2022

An official PyTorch implementation of the TKDE paper "Self-Supervised Graph Representation Learning via Topology Transformations".

Self-Supervised Graph Representation Learning via Topology Transformations This repository is the official PyTorch implementation of the following pap

2 Oct 31, 2022

[CVPR2021] The source code for our paper 《Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning》.

TBE The source code for our paper "Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Le

150 Dec 28, 2022

[AAAI2021] The source code for our paper 《Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion》.

DSM The source code for paper Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion Project Website; Datasets li

114 Oct 16, 2022

This is the pytorch code for the paper Curious Representation Learning for Embodied Intelligence.

Related tags

Overview

Curious Representation Learning for Embodied Intelligence

Interactive Pretraining of Embodied Agents

Downstream ImageNav Pretraining

Downstream ObjectNav Pretraining

Pretrained Weights

Citing Our Paper

You might also like...

A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

This is the official pytorch implementation for the paper: Instance Similarity Learning for Unsupervised Feature Representation.

An official PyTorch implementation of the TKDE paper "Self-Supervised Graph Representation Learning via Topology Transformations".

[CVPR2021] The source code for our paper 《Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learning》.

Implementation of the paper "Language-agnostic representation learning of source code from structure and context".

code for paper "Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?"

This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

[AAAI2021] The source code for our paper 《Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motion》.

Comments

ObjNav MP3D-Val Results

Code to compute area of coverage from different RL agent

Owner

Kroomsa: A search engine for the curious

PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

School of Artificial Intelligence at the Nanjing University (NJU)School of Artificial Intelligence at the Nanjing University (NJU)

Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

Megaverse is a new 3D simulation platform for reinforcement learning and embodied AI research

The Surprising Effectiveness of Visual Odometry Techniques for Embodied PointGoal Navigation

YouRefIt: Embodied Reference Understanding with Language and Gesture

PyTorch implementation code for the paper MixCo: Mix-up Contrastive Learning for Visual Representation

Eff video representation - Efficient video representation through neural fields

Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images