Learning multiple gaits of quadruped robot using hierarchical reinforcement learning

Last update: Dec 11, 2022

Overview

Learning multiple gaits of quadruped robot using hierarchical reinforcement learning

We propose a method to learn multiple gaits of quadruped robot using hierarchical reinforcement learning. We designed a hierarchical controller and a learning framework that could learn and show multiple gaits in a single policy. Every experiment was done in RAISIM simulator link.

Using our method, quadruped robot can learn multiple gaits including Trot, Pace, and Bound(imperfect). We further successfully learned multiple gait in a single policy using our framework. To show the existence of optimal gaits for specific velocity range, we held an analysis of mechanical energy usage for each learned gaits. Check the paper for detailed results.

Paper: http://arxiv.org/abs/2112.04741
Project slides: https://bit.ly/3ADOjV1

Method

Result

Trot

2. Pace

3. Bound (imperfect)

4. Multiple gaits in a single policy (Pace & Bound)

Cite

@inproceedings{kim2021multiplegait,
    author = {Kim, Yunho and Son, Bukun and Lee, Dongjun},
    title = {Learning multiple gaits of quadruped robot using hierarchical reinforcement learning},
    booktitle={http://arxiv.org/abs/2112.04741},
    year={2021}
}

Comments

I seriously doubt the rationality of the method used in the article.

First, only the reference motion of the thigh joint is used in the experiment. According to the characteristics of reinforcement learning, legal multi-gait motion can not be obtained.

Second, the CPG signal is not used as the input of the strategy network in the experiment, but only the phase parameter is input. Using only one hot coding is not enough to help the strategy network successfully learn different gait.

What can prove this is that the URDF file is not provided in the code, and using the code in the code warehouse can not get the effect shown in the article.

opened by chillybird 6

Robot Reinforcement Learning on the Constraint Manifold

Implementation of "Robot Reinforcement Learning on the Constraint Manifold"

31 Dec 5, 2022

Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX

CQL-JAX This repository implements Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX (FLAX). Implementation is built on

8 Nov 7, 2022

Reinforcement-learning - Repository of the class assignment questions for the course on reinforcement learning

DSE 314/614: Reinforcement Learning This repository containing reinforcement lea

4 Apr 15, 2022

Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts)

Introduction Pytorch implementation of Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Expert. | paper Song Park1

97 Dec 23, 2022

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers

hierarchical-transformer-1d Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers In Progress!! 2021.

7 Nov 6, 2022

A custom-designed Spider Robot trained to walk using Deep RL in a PyBullet Simulation

SpiderBot_DeepRL Title: Implementation of Single and Multi-Agent Deep Reinforcement Learning Algorithms for a Walking Spider Robot Authors(s): Arijit

9 Jul 28, 2022

Control-Robot-Arm-using-PS4-Controller - A Robotic Arm based on Raspberry Pi and Arduino that controlled by PS4 Controller

Control-Robot-Arm-using-PS4-Controller You can see all details about this Robot

5 Jan 1, 2022

Code for: Gradient-based Hierarchical Clustering using Continuous Representations of Trees in Hyperbolic Space. Nicholas Monath, Manzil Zaheer, Daniel Silva, Andrew McCallum, Amr Ahmed. KDD 2019.

gHHC Code for: Gradient-based Hierarchical Clustering using Continuous Representations of Trees in Hyperbolic Space. Nicholas Monath, Manzil Zaheer, D

35 Nov 16, 2022

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Swin Transformer for Object Detection This repo contains the supported code and configuration files to reproduce object detection results of Swin Tran

1.4k Dec 30, 2022

Learning multiple gaits of quadruped robot using hierarchical reinforcement learning

Related tags

Overview

Learning multiple gaits of quadruped robot using hierarchical reinforcement learning

Method

Result

Cite

You might also like...

Robot Reinforcement Learning on the Constraint Manifold

Conservative Q Learning for Offline Reinforcement Reinforcement Learning in JAX

Reinforcement-learning - Repository of the class assignment questions for the course on reinforcement learning

Official PyTorch implementation of MX-Font (Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts)

Implementation of H-Transformer-1D, Hierarchical Attention for Sequence Learning using 🤗 transformers

A custom-designed Spider Robot trained to walk using Deep RL in a PyBullet Simulation

Control-Robot-Arm-using-PS4-Controller - A Robotic Arm based on Raspberry Pi and Arduino that controlled by PS4 Controller

Code for: Gradient-based Hierarchical Clustering using Continuous Representations of Trees in Hyperbolic Space. Nicholas Monath, Manzil Zaheer, Daniel Silva, Andrew McCallum, Amr Ahmed. KDD 2019.

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Comments

I seriously doubt the rationality of the method used in the article.

Owner

Yunho Kim

Control-Raspberry-Pi-Robot-using-Hand-Gestures - A 4WD Robot car based on Raspberry Pi that controlled by hand gestures(using openCV and mediapipe)

Official implementation of "Learning Forward Dynamics Model and Informed Trajectory Sampler for Safe Quadruped Navigation" (RSS 2022)

Space robot - (Course Project) Using the space robot to capture the target satellite that is disabled and spinning, then stabilize and fix it up

Guiding evolutionary strategies by (inaccurate) differentiable robot simulators @ NeurIPS, 4th Robot Learning Workshop

Anomaly Detection Based on Hierarchical Clustering of Mobile Robot Data

Hierarchical-Bayesian-Defense - Towards Adversarial Robustness of Bayesian Neural Network through Hierarchical Variational Inference (Openreview)

[IROS'21] SurRoL: An Open-source Reinforcement Learning Centered and dVRK Compatible Platform for Surgical Robot Learning

[ICML 2020] Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control

Motion Planner Augmented Reinforcement Learning for Robot Manipulation in Obstructed Environments (CoRL 2020)

Deep Reinforcement Learning for mobile robot navigation in ROS Gazebo simulator