Data Preparation, Processing, and Visualization for MoVi Data

Related tags

Deep Learning MoVi-Toolbox
Overview

MoVi-Toolbox

Data Preparation, Processing, and Visualization for MoVi Data, https://www.biomotionlab.ca/movi/

MoVi is a large multipurpose dataset of human motion and video.

Here we provide tools and tutorials to use MoVi in your research projects. More specifically:

Table of Contents

Installation

Requirements

  • Python 3.*
  • MATLAB v>2017

In case you are interested in using body shape data (or also AMASS/MoVi original data) follow the instructions on AMASS Github page.

Tutorials

  • We have provided very brief tutorials on how to use the dataset in MoCap. Some of the functions are only provided in MATLAB or Python so please take a look at both tutorial files tutorial_MATLAB.m and tutorial_python.ipynb.

  • The tutorial on how to have access to the dataset is given here.

Important Notes

  • The video data for each round are provided as a single sequence (and not individual motions). In case you are interested in having synchronized video and AMASS (joint and body) data, you should trim F_PGx_Subject_x_L.avi files into single motion video files using single_videos.m function.
  • The timestamps (which separate motions) are provided by the name of “flags” in V3D files (only for f and s rounds). Please notice that “flags30” can be used for video data and “flags120” can be used for mocap data. The reason for having two types of flags is that video data were recorded in 30 fps and mocap data were recorded in 120 fps.
  • The body mesh is not provided in AMASS files by default. Please use amass_fk function to augment AMASS data with the corresponding body mesh (vertices). (the details are explained in the tutorial_python.ipynb)

Citation

Please cite the following paper if you use this code directly or indirectly in your research/projects:

@misc{ghorbani2020movi,
    title={MoVi: A Large Multipurpose Motion and Video Dataset},
    author={Saeed Ghorbani and Kimia Mahdaviani and Anne Thaler and Konrad Kording and Douglas James Cook and Gunnar Blohm and Nikolaus F. Troje},
    year={2020},
    eprint={2003.01888},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

License

Software Copyright License for non-commercial scientific research purposes. Before you download and/or use the Motion and Video (MoVi) dataset, please carefully read the terms and conditions stated on our website and in any accompanying documentation. If you are using the part of the dataset that was post-processed as part of AMASS, you must follow all their terms and conditions as well. By downloading and/or using the data or the code (including downloading, cloning, installing, and any other use of this GitHub repository), you acknowledge that you have read these terms and conditions, understand them, and agree to be bound by them. If you do not agree with these terms and conditions, you must not download and/or use the MoVi dataset and any associated code and software. Any infringement of the terms of this agreement will automatically terminate your rights under this License.

Contact

The code in this repository is developed by Saeed Ghorbani.

If you have any questions you can contact us at [email protected].

Comments
  • Mean limb offset and FK script

    Mean limb offset and FK script

    Hi Saeed,

    Where can I find the mean limb offsets and did you guys decide to make public an FK script to convert the Exp maps to the join locations?

    Thanks.

    opened by alecda573 5
  • Question regarding F_Subject_1_L1.avi

    Question regarding F_Subject_1_L1.avi

    Hey Saeed,

    Could you tell me from where I can download F_Subject_1_L1.avi file and similar ones. I can't find it in https://www.biomotionlab.ca/movi/

    BR, Ugnius

    opened by ugnelis 4
  • How to project marker data on CP1/2 videos?

    How to project marker data on CP1/2 videos?

    First of all, thanks for this useful dataset! Can you please explain how to project marker/joint/keypoint data onto the CP1/CP2 videos? I can only find calibration data for PG1/PG2 videos, which have lower quality.

    opened by isarandi 3
  • Matlab example doesn't work on Linux

    Matlab example doesn't work on Linux

    Hi,

    I have Matlab 2017a, and I cannot run the tutorial. Matlab says that .mlx files are not supported on Linux.

    Is it possible to get this example in Python? image

    Best Regards, Ugnius

    opened by ugnelis 2
  • About S1, S2 round GT pose

    About S1, S2 round GT pose

    Hi, i wanna know whether there are GT poses for S1, S2 round. I can only find the IMU results for them. And can you provide the raw IMU data for S1, S2 round? thanks.

    opened by cgang16 0
  • Removed model_type key.

    Removed model_type key.

    Fixed error:

    ---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-cee14851a8b1> in <module>
      2 bm_path = '/home/ugnelis/GIT/MoVi/chark/amass/body_models/smplh/male/model.npz'
      3 
----> 4 joints, verts = amass_fk(npz_bdata_path, bm_path)
      5 print('Shape of joints tensor: ', joints.shape)
      6 print('Shape of verts tensor: ', verts.shape)

~/GIT/MoVi/chark/MoVi-Toolbox/MoCap/utils.py in amass_fk(npz_data_path, bm_path)
    133     else:
    134         comp_device = torch.device("cpu")
--> 135     bm = BodyModel(bm_path=bm_path, model_type="smplh", batch_size=1, num_betas=10).to(
    136         comp_device
    137     )

TypeError: __init__() got an unexpected keyword argument 'model_type'

    The bug was in utils.py. This files was sending flag model_type="smplh" via BodyModel but the BodyModel wasn't expecting (https://github.com/nghorbani/human_body_prior/blob/master/human_body_prior/body_model/body_model.py#L35). The model type itself is defined automatically without passing a flag.

    opened by ugnelis 0
Owner
Saeed Ghorbani
Graduate student in EECS department at York University
Saeed Ghorbani
GitHub https://www.biomotionlab.ca/movi
Web mining module for Python, with tools for scraping, natural language processing, machine learning, network analysis and visualization.

Pattern Pattern is a web mining module for Python. It has tools for: Data Mining: web services (Google, Twitter, Wikipedia), web crawler, HTML DOM par

Computational Linguistics Research Group 8.4k Jan 3, 2023
Third party Pytorch implement of Image Processing Transformer (Pre-Trained Image Processing Transformer arXiv:2012.00364v2)

ImageProcessingTransformer Third party Pytorch implement of Image Processing Transformer (Pre-Trained Image Processing Transformer arXiv:2012.00364v2)

null 61 Jan 1, 2023
Team nan solution repository for FPT data-centric competition. Data augmentation, Albumentation, Mosaic, Visualization, KNN application

FPT_data_centric_competition - Team nan solution repository for FPT data-centric competition. Data augmentation, Albumentation, Mosaic, Visualization, KNN application

Pham Viet Hoang (Harry) 2 Oct 30, 2022
QuakeLabeler is a Python package to create and manage your seismic training data, processes, and visualization in a single place — so you can focus on building the next big thing.

QuakeLabeler Quake Labeler was born from the need for seismologists and developers who are not AI specialists to easily, quickly, and independently bu

Hao Mai 15 Nov 4, 2022
PyMove is a Python library to simplify queries and visualization of trajectories and other spatial-temporal data

Use PyMove and go much further Information Package Status License Python Version Platforms Build Status PyPi version PyPi Downloads Conda version Cond

Insight Data Science Lab 64 Nov 15, 2022
Pytorch Implementations of large number classical backbone CNNs, data enhancement, torch loss, attention, visualization and some common algorithms.

Torch-template-for-deep-learning Pytorch implementations of some **classical backbone CNNs, data enhancement, torch loss, attention, visualization and

Li Shengyan 270 Dec 31, 2022
Data visualization app for H&M competition in kaggle

handm_data_visualize_app Data visualization app by streamlit for H&M competition in kaggle. competition page: https://www.kaggle.com/competitions/h-an

Kyohei Uto 12 Apr 30, 2022
MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

Microsoft 5.7k Jan 9, 2023
Leaderboard and Visualization for RLCard

RLCard Showdown This is the GUI support for the RLCard project and DouZero project. RLCard-Showdown provides evaluation and visualization tools to hel

Data Analytics Lab at Texas A&M University 246 Dec 26, 2022
PyTorch implementation of Value Iteration Networks (VIN): Clean, Simple and Modular. Visualization in Visdom.

VIN: Value Iteration Networks This is an implementation of Value Iteration Networks (VIN) in PyTorch to reproduce the results.(TensorFlow version) Key

Xingdong Zuo 215 Dec 7, 2022
A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

Website, Tutorials, and Docs    Uncertainty Toolbox A python toolbox for predictive uncertainty quantification, calibration, metrics, and visualizatio

Uncertainty Toolbox 1.4k Dec 28, 2022
Interactive Terraform visualization. State and configuration explorer.

Rover - Terraform Visualizer Rover is a Terraform visualizer. In order to do this, Rover: generates a plan file and parses the configuration in the ro

Tu Nguyen 2.3k Jan 7, 2023
Reporting and Visualization for Hazardous Events

Reporting and Visualization for Hazardous Events

Jv Kyle Eclarin 2 Oct 3, 2021
Doosan robotic arm, simulation, control, visualization in Gazebo and ROS2 for Reinforcement Learning.

Robotic Arm Simulation in ROS2 and Gazebo General Overview This repository includes: First, how to simulate a 6DoF Robotic Arm from scratch using GAZE

David Valencia 12 Jan 2, 2023
Neurons Dataset API - The official dataloader and visualization tools for Neurons Datasets.

Neurons Dataset API - The official dataloader and visualization tools for Neurons Datasets. Introduction We propose our dataloader API for loading and

null 1 Nov 19, 2021
Clustergram - Visualization and diagnostics for cluster analysis in Python

Clustergram Visualization and diagnostics for cluster analysis Clustergram is a diagram proposed by Matthias Schonlau in his paper The clustergram: A

Martin Fleischmann 96 Dec 26, 2022
Optimizers-visualized - Visualization of different optimizers on local minimas and saddle points.

Optimizers Visualized Visualization of how different optimizers handle mathematical functions for optimization. Contents Installation Usage Functions

Gautam J 1 Jan 1, 2022
Company clustering with K-means/GMM and visualization with PCA, t-SNE, using SSAN relation extraction

RE results graph visualization and company clustering Installation pip install -r requirements.txt python -m nltk.downloader stopwords python3.7 main.

Jieun Han 1 Oct 6, 2022
Visualization toolkit for neural networks in PyTorch! Demo -->

FlashTorch A Python visualization toolkit, built with PyTorch, for neural networks in PyTorch. Neural networks are often described as "black box". The

Misa Ogura 692 Dec 29, 2022