Neural Motion Learner With Python

Jinseok Bae

Last update: Nov 28, 2022

Related tags

Deep Learning neural_motion_learner

Overview

Neural Motion Learner

Introduction

This work is to extract skeletal structure from volumetric observations and to learn motion dynamics from the detected skeletal motions in a fully unsupervised manner.

Our model conducts motion generation/interpolation/retargeting based on the learned latent dynamics.

Note that it is an unofficial version of the work so that minimal amounts of codes are provided to demonstrate results.

Full descriptions including title, training codes and data pre-processing methods will be uploaded once the paper of this work is accepted to the conference.

Install

We tested on Python 3.8 and Ubuntu 18.04 LTS.

The architecture is built from Pytorch 1.7.1 with Cuda 11.0.

Creating a conda environment is recommended.

## Download the repository
git clone https://github.com/jinseokbae/neural_motion_learner.git
cd neural_motion_learner
## Create conda env
conda create --name nmotion python=3.8
conda activate nmotion
## modify setup.sh to match your cuda setting
bash setup.sh

Run

Using provided pretrained model, run demo codes to visualize followings:

## Motion generation
python vis_generation.py
## Result will be stored in output/generation

## Motion interpolation
python vis_interpolation.py
## Result will be stored in output/interpolation

## Motion retargeting
python vis_retarget.py
## Result will be stored in output/retarget

You might also like...

EasyMocap is an open-source toolbox for markerless human motion capture from RGB videos.

Comments

Dataset Generation Process

Hello and thank you for your very interesting work!

I wanted to ask you, could you please provide us with the dataset generation process? I am mainly interested in the DeformingThings4D splits, and I can see that you are loading numpy arrays. How are you generating these arrays? What is the folder structure?

Best, Konstantinos

opened by ktertikas 2
Pretrained weights for DeformingThings4D and Robot

Thank you, for releasing the code for your amazing paper. I have to perform experiments on non-human sequences like animals and robot. Can you kindly share the trained weights for them as well.

opened by shubhMaheshwari 1

Neural Motion Learner With Python

Related tags

Overview

Neural Motion Learner

Introduction

Install

Run

You might also like...

EasyMocap is an open-source toolbox for markerless human motion capture from RGB videos.

Object Depth via Motion and Detection Dataset

Character Controllers using Motion VAEs

dataset for ECCV 2020 "Motion Capture from Internet Videos"

[arXiv] What-If Motion Prediction for Autonomous Driving ❓🚗💨

Learning to Estimate Hidden Motions with Global Motion Aggregation

Synthesizing Long-Term 3D Human Motion and Interaction in 3D in CVPR2021

PyTorch implementation DRO: Deep Recurrent Optimizer for Structure-from-Motion

[WACV 2020] Reducing Footskate in Human Motion Reconstruction with Ground Contact Constraints

Comments

Dataset Generation Process

Pretrained weights for DeformingThings4D and Robot

Owner

Jinseok Bae

This repo contains the pytorch implementation for Dynamic Concept Learner (accepted by ICLR 2021).

The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task —— Next Sentence Prediction"

The Self-Supervised Learner can be used to train a classifier with fewer labeled examples needed using self-supervised learning.

PyTorch implementation for the visual prior component (i.e. perception module) of the Visually Grounded Physics Learner [Li et al., 2020].

This repository contains the code for the paper "Hierarchical Motion Understanding via Motion Programs"

Exploring Versatile Prior for Human Motion via Motion Frequency Guidance (3DV2021)

This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

Kaggle Lyft Motion Prediction for Autonomous Vehicles 4th place solution

data/code repository of "C2F-FWN: Coarse-to-Fine Flow Warping Network for Spatial-Temporal Consistent Motion Transfer"

PaddleRobotics is an open-source algorithm library for robots based on Paddle, including open-source parts such as human-robot interaction, complex motion control, environment perception, SLAM positioning, and navigation.