Awesome-Human-Pose-Prediction

A selection of State Of The Art research papers (and code) on human trajectory prediction (forecasting). Papers marked with [W] are workshop papers.

Maintainers: Karttikeya Mangalam

Contributing: Please feel free to pull requests to add new resources or suggest addditions or changes to the list. While proposing a new addition, please keep in mind the following principles:

• The work has been accepted in a reputable peer reviewed publication venue.
• An opensource link to the paper pdf is attached (as far as possible).
• Code for the paper is linked (if made opensource by the authors).

Email: mangalam@cs.{berkeley,stanford).edu

Datasets

• Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments [Paper]
• Stanford Drone Dataset (SDD): Learning Social Etiquette: Human Trajectory Understanding in Crowded Scenes [Paper] [Leaderboard]

Papers

As End in Itself

• From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting [Paper]

• It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction [Paper]

• Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data [Paper]

• Interaction-Based Trajectory Prediction Over a Hybrid Traffic Graph [paper]

• Map-Adaptive Goal-Based Trajectory Prediction [paper]

• Interaction-Aware Trajectory Prediction based on a 3D Spatio-Temporal Tensor Representation using Convolutional–Recurrent Neural Networks [paper]

• DROGON: A Trajectory Prediction Model based on Intention-Conditioned Behavior Reasoning [Paper]

• Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction [Paper]

• Social-VRNN: One-Shot Multi-modal Trajectory Prediction for Interacting Pedestrians [Paper]

• Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions [Paper]

• Social NCE: Contrastive Learning of Socially-aware Motion Representations [Paper]

• Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach [Paper]

• Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction [Paper]

• Deep Learning for Vision-based Prediction: A Survey [Paper]

• Probabilistic Crowd GAN: Multimodal Pedestrian Trajectory Prediction Using a Graph Vehicle-Pedestrian Attention Network [Paper]

• Semantics for Robotic Mapping, Perception and Interaction: A Survey [Paper]

• Benchmark for Evaluating Pedestrian Action Prediction[Paper]

• Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking [Paper]

• Pedestrian Behavior Prediction via Multitask Learning and Categorical Interaction Modeling [Paper]

• Graph-SIM: A Graph-based Spatiotemporal Interaction Modelling for Pedestrian Action Prediction [Paper]

• Haar Wavelet based Block Autoregressive Flows for Trajectories [Paper]

• Imitative Planning using Conditional Normalizing Flow [Paper]

• TNT: Target-driveN Trajectory Prediction [Paper]

• SimAug: Learning Robust Representations from Simulation for Trajectory Prediction [Paper]

• SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints [Paper]

• Social GAN: Socially Acceptable Trajectories With Generative Adversarial Networks [Paper]

• DESIRE: Distant Future Prediction in Dynamic Scenes With Interacting Agents [Paper]

• Predicting Whole Body Motion Trajectories using Conditional Neural Movement Primitives [Paper] [W]

• Anticipating Human Intention for Full-Body Motion Prediction [Paper] [W]

• Human Motion Prediction With Graph Neural Networks [Paper] [W]

• Action-Agnostic Human Pose Forecasting [Paper]

• Human Torso Pose Forecasting in the Real World [Paper]

• Imitation Learning for Human Pose Prediction [Paper]

• Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision [Paper]

• Predicting 3D Human Dynamics from Video [Paper]

• Recurrent Network Models for Human Dynamics [Paper]

• Structural-RNN: Deep Learning on Spatio-Temporal Graphs [Paper]

• Learning Trajectory Dependencies for Human Motion Prediction [Paper]

• Anticipating many futures: Online human motion prediction and generation for human-robot interaction [Paper]

• Teaching Robots to Predict Human Motion [Paper]

• Deep representation learning for human motion prediction and classification [Paper]

• On human motion prediction using recurrent neural networks [Paper]

• Few-Shot Human Motion Prediction via Meta-learning [Paper]

• Efficient convolutional hierarchical autoencoder for human motion prediction [Paper]

• Learning Human Motion Models for Long-term Predictions [Paper]

• Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic [Paper]

• Context-aware Human Motion Prediction [Paper]

• Adversarial Geometry-Aware Human Motion Prediction [Paper]

• Convolutional Sequence to Sequence Model for Human Dynamics [Paper]

• QuaterNet: A Quaternion-based Recurrent Model for Human Motion [Paper]

• BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN [Paper]

• Human Motion Modeling using DVGANs [Paper]

• Human Motion Prediction using Semi-adaptable Neural Networks [Paper]

• A Neural Temporal Model for Human Motion Prediction [Paper]

• Modeling Human Motion with Quaternion-based Neural Networks [Paper]

• Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies [Paper]

• VRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction [Paper]

• EAN: Error Attenuation Network for Long-term Human Motion Prediction [Paper]

• Structured Prediction Helps 3D Human Motion Modelling [Paper]

• Forecasting Human Dynamics from Static Images [Paper]

• HP-GAN: Probabilistic 3D human motion prediction via GAN [Paper]

• Learning Latent Representations of 3D Human Pose with Deep Neural Networks [Paper]

• A Recurrent Variational Autoencoder for Human Motion Synthesis [Paper]

• Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling [Paper]

• Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control [Paper]

• PISEP2: Pseudo Image Sequence Evolution based 3D Pose Prediction [Paper]

• Human Motion Prediction via Spatio-Temporal Inpainting [Paper]

• Spatiotemporal Co-attention Recurrent Neural Networks for Human-Skeleton Motion Prediction [Paper]

• Human Pose Forecasting via Deep Markov Models [Paper]

• Auto-Conditioned Recurrent Networks For Extended Complex Human Motion Synthesis [Paper]

• Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network [Paper]

As a Subtask

• The Pose Knows: Video Forecasting by Generating Pose Futures [Paper]
• I-Planner: Intention-Aware Motion Planning Using Learning Based Human Motion Prediction [Paper]
• Language2Pose: Natural Language Grounded Pose Forecasting [Paper]
• Long-Term Video Generation of Multiple Futures Using Human Poses [Paper]
• Predicting body movements for person identification under different walking conditions [Paper]