The code written during my Bachelor Thesis "Classification of Human Whole-Body Motion using Hidden Markov Models".

Overview

This code was written during the course of my Bachelor thesis Classification of Human Whole-Body Motion using Hidden Markov Models. Some things might be broken and I definitely don't recommend to use any of the code in any sort of production application. However, for research purposes this code might be useful so I decided to open-source it. Use at your own risk!

Requirements

Use pip to install most requriements (pip install -r requriements.txt). Sometimes this causes problems if Cython, numpy and scipy are not already installed, in which case this needs to be done manually.

Additionally, some packages must be installed that are not provided by pip.

pySimox and pyMMM

pySimox and pyMMM must be installed manually as well. To build them, perform the following steps:

git submodule update --init --recursive
cd vendor/pySimox/build
cmake ..
make
cp _pysimox.so ../../../lib/python2.7/site-packages/_pysimox.so
cp pysimox.py ../../../lib/python2.7/site-packages/pysimox.py
cd ../pyMMM/build
cmake ..
make
cp _pymmm.so ../../../lib/python2.7/site-packages/_pymmm.so
cp pymmm.py ../../../lib/python2.7/site-packages/pymmm.py

Note that the installation script may need some fine-tuning. Additionally, this assumes that all virtualenv is set up in the root of this git repo.

Basic Usage

This repo contains two main programs: dataset.py and evaluate_new.py. All of them are located in src and should be run from this directory. There are some additional files in there, some of them are out-dated and should be deleted (e.g. evaluate.py), some of them are really just scripts and should be moved to the scripts folder eventually.

The dataset tool

The dataset tool is concerened with handling everything related to datasets: plot plots features, export saves a dataset in a variety of formats, report prints details about a dataset and check performs a consistency check. Additionally, export-all can be used to create a dataset that contains all features (normalized and unnormalized) by merging Vicon C3D and MMM files into one giant file. A couple of examples:

  • python dataset.py ../data/dataset1.json plot --features root_pos plots the root_pos feature of all motions in the dataset; the dataset can be a JSON manifest or a pickled dataset
  • python dataset.py ../data/dataset1.json export --output ~/export.pkl exports dataset1 as a single pickled file; usually a JSON manifest is used
  • python dataset.py ../data/dataset1.json export-all --output ~/export_all.pkl exports dataset1 by combining vicon and MMM files and by computing both the normalized and unnormalized version of all features. It also performs normalization on the vicon data by using additional information from the MMM data (namely the root_pos and root_rot); the dataset has to be a JSON manifest
  • python dataset.py ../data/dataset1.json report prints details about a dataset; the dataset can be a JSON manifest or a pickled dataset
  • python dataset.py ../data/dataset1.json check performs a consistency check of a dataset; the manifest has to be a JSON manifest

Additional parameters are avaialble for most commands. Use dataset --help to get an overview.

The evaluate_new tool

The evaluate_new tool can be used to perform feature selection (using the feature command) or to evaluate different types of models with decision makers (by using the model command). It is important to note that the evaluate_new tool expects a pickled version of the dataset, hence export or export_all must be used to prepare a dataset. This is to avoid the computational complexity.

A couple of examples:

  • python evaluate_new.py model ../data/export_all.pkl --features normalized_joint_pos normalized_root_pos --decision-maker log-regression --n-states 5 --model fhmm-seq --output-dir ~/out trains a HMM ensemble with each HMM having 5 states on the normalized_joint_pos and normalized_root_pos features and uses logistic regression to perform the final predicition. The results are also saved in the directory ~/out
  • python evaluate_new.py features ../data/export_all.pkl --features normalized_joint_pos normalized_root_pos --measure wasserstein performs feature selection using the starting set normalized_joint_pos normalized_root_pos and the wasserstein measure

From dataset to result

First, define a JSON manifest dataset.json that links together the individual motions and pick labels. Next, export the dataset by using python dataset.py ../data/dataset.json export-all --output ../data/dataset_all.pkl. If you need smoothing, simply load the dataset (using pickle.load()), call smooth_features() on the Dataset object and dump it to a new file. There's currently no script for this but it can be done using three lines and the interactive python interpreter. Next, perform feature selection using python evaluate_new.py features ../data/dataset_all.pkl --features <list of features> --measure wasserstein --output-dir ~/features --transformers minmax-scaler. You'll want to use the minmax scaler transformer to avoid numerical problems during training. This will probably take a while. The results (at ~/features) will give you the best feature subsets that were found. Next, use those features to train an HMM ensemble: python evaluate_new model ../data/dataset_all.pkl --features <best features> --model fhmm-seq --n-chains 2 --n-states 10 --n-training-iter 30 -decision-maker log-regression --transformers minmax-scaler --output-dir ~/train (again, the minmax-scaler is almost always a good idea). The results will be in ~/output.

You might also like...
Pytorch reimplement of the paper "A Novel Cascade Binary Tagging Framework for Relational Triple Extraction" ACL2020. The original code is written in keras.

CasRel-pytorch-reimplement Pytorch reimplement of the paper "A Novel Cascade Binary Tagging Framework for Relational Triple Extraction" ACL2020. The o

Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Regularized Greedy Forest Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better r

🎯 A comprehensive gradient-free optimization framework written in Python
🎯 A comprehensive gradient-free optimization framework written in Python

Solid is a Python framework for gradient-free optimization. It contains basic versions of many of the most common optimization algorithms that do not

Minimal deep learning library written from scratch in Python, using NumPy/CuPy.
Minimal deep learning library written from scratch in Python, using NumPy/CuPy.

SmallPebble Project status: experimental, unstable. SmallPebble is a minimal/toy automatic differentiation/deep learning library written from scratch

A parametric soroban written with CADQuery.
A parametric soroban written with CADQuery.

A parametric soroban written in CADQuery The purpose of this project is to demonstrate how "code CAD" can be intuitive to learn. See soroban.py for a

FluidNet re-written with ATen tensor lib
FluidNet re-written with ATen tensor lib

fluidnet_cxx: Accelerating Fluid Simulation with Convolutional Neural Networks. A PyTorch/ATen Implementation. This repository is based on the paper,

The Noise Contrastive Estimation for softmax output written in Pytorch
The Noise Contrastive Estimation for softmax output written in Pytorch

An NCE implementation in pytorch About NCE Noise Contrastive Estimation (NCE) is an approximation method that is used to work around the huge computat

A plug-and-play library for neural networks written in Python
A plug-and-play library for neural networks written in Python

A plug-and-play library for neural networks written in Python!

Some toy examples of score matching algorithms written in PyTorch
Some toy examples of score matching algorithms written in PyTorch

toy_gradlogp This repo implements some toy examples of the following score matching algorithms in PyTorch: ssm-vr: sliced score matching with variance

Owner
Matthias Plappert
I am a research scientist working on machine learning, and especially deep reinforcement learning, in robotics.
Matthias Plappert
Code for the bachelors-thesis flaky fault localization

Flaky_Fault_Localization Scripts for the Bachelors-Thesis: "Flaky Fault Localization" by Christian Kasberger. The thesis examines the usefulness of sp

Christian Kasberger 1 Oct 26, 2021
Implementation of the master's thesis "Temporal copying and local hallucination for video inpainting".

Temporal copying and local hallucination for video inpainting This repository contains the implementation of my master's thesis "Temporal copying and

David Álvarez de la Torre 1 Dec 2, 2022
Politecnico of Turin Thesis: "Implementation and Evaluation of an Educational Chatbot based on NLP Techniques"

THESIS_CAIRONE_FIORENTINO Politecnico of Turin Thesis: "Implementation and Evaluation of an Educational Chatbot based on NLP Techniques" GENERATE TOKE

cairone_fiorentino97 1 Dec 10, 2021
Training code and evaluation benchmarks for the "Self-Supervised Policy Adaptation during Deployment" paper.

Self-Supervised Policy Adaptation during Deployment PyTorch implementation of PAD and evaluation benchmarks from Self-Supervised Policy Adaptation dur

Nicklas Hansen 101 Nov 1, 2022
Pose Detection and Machine Learning for real-time body posture analysis during exercise to provide audiovisual feedback on improvement of form.

Posture: Pose Tracking and Machine Learning for prescribing corrective suggestions to improve posture and form while exercising. This repository conta

Pratham Mehta 10 Nov 11, 2022
RoBERTa Marathi Language model trained from scratch during huggingface 🤗 x flax community week

RoBERTa base model for Marathi Language (मराठी भाषा) Pretrained model on Marathi language using a masked language modeling (MLM) objective. RoBERTa wa

Nipun Sadvilkar 23 Oct 19, 2022
PyTorchMemTracer - Depict GPU memory footprint during DNN training of PyTorch

A Memory Tracer For PyTorch OOM is a nightmare for PyTorch users. However, most

Jiarui Fang 9 Nov 14, 2022
Compare outputs between layers written in Tensorflow and layers written in Pytorch

Compare outputs of Wasserstein GANs between TensorFlow vs Pytorch This is our testing module for the implementation of improved WGAN in Pytorch Prereq

Hung Nguyen 72 Dec 20, 2022
Code for all the Advent of Code'21 challenges mostly written in python

Advent of Code 21 Code for all the Advent of Code'21 challenges mostly written in python. They are not necessarily the best or fastest solutions but j

null 4 May 26, 2022
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 66 Dec 26, 2022