Web service for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation based on OpenFace 2.0

Overview

OpenGaze: Web Service for OpenFace Facial Behaviour Analysis Toolkit

Python Badge FastAPI

Overview

OpenFace is a fantastic tool intended for computer vision and machine learning researchers, the affective computing community, and people interested in building interactive applications based on facial behavior analysis. OpenFace is the first toolkit capable of facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation with available source code for running and training the models.

Even though the documentation of this tool is very detailed and organized, it is still problematic to use in web services because of its C++ codebase and CLI interface.

OpenGaze is a single endpoint RESTful web API service with HTTP Basic Authentication developed with FastAPI framework that uses the FaceLandmarkImg executable of OpenFace and provides a web API that responds with crucial eye-gaze and head-pose related fields.

screenshot

Installation

OpenFace Installation

Follow the installation instruction of OpenFace to install it first with all the dependencies. Platform based installation instructions are as follows-

OS Installation Instruction
Unix-like View Wiki
Windows View Wiki
Mac View Wiki

The install script for Unix-like systems only work for Debian-based systems including ubuntu. For other distros, please install the dependencies manually and then compile the code following the Advanced Ubuntu Installation section.

At the end of third step of Actual OpenFace installation following command can be used to install the executables in the system.

make install

If make install is not used, FACE_LANDMARK_IMG_EXEC_COMMAND environment variable in .env file will need to be updated with the absolute path of the FaceLandmarkImg executable. e.g., /home/user/OpenFace/build/bin/FaceLandmarkImg

Run OpenGaze

Clone this repo

git clone https://github.com/nsssayom/OpenGaze.git
cd OpenGaze

Install virtualenv package for your OS. For Debian-based systems-

sudo apt update
sudo apt install python3-virtualenv

Create and activate a virtual environment.

virtualenv env
source env/bin/activate

Install dependencies using pip.

pip install -r requirements.txt

Run OpenGaze

uvicorn main:app --reload 

Usages

Now, test the API with any API testing tool. cURL snippet is following

curl --request POST \
  --url http://localhost:8000/ \
  --header 'Authorization: Basic <RandomHeader>' \
  --header 'Content-Type: application/json' \
  --data '{
	"image_url": "https://wamu.org/wp-content/uploads/2019/05/Sun-article.jpeg"
}'

use API_KEY value in .env file as username and API Secret as password for basic authentication.

You might also like...
Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21).

ACTION-Net Official implementation of ACTION-Net: Multipath Excitation for Action Recognition (CVPR'21). Getting Started EgoGesture data folder struct

Video lie detector using xgboost - A video lie detector using OpenFace and xgboost

video_lie_detector_using_xgboost a video lie detector using OpenFace and xgboost

SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation
SE3 Pose Interp - Interpolate camera pose or trajectory in SE3, pose interpolation, trajectory interpolation

SE3 Pose Interpolation Pose estimated from SLAM system are always discrete, and

PyTorch module to use OpenFace's nn4.small2.v1.t7 model

OpenFace for Pytorch Disclaimer: This codes require the input face-images that are aligned and cropped in the same way of the original OpenFace. * I m

The codes and models in 'Gaze Estimation using Transformer'.
The codes and models in 'Gaze Estimation using Transformer'.

GazeTR We provide the code of GazeTR-Hybrid in "Gaze Estimation using Transformer". We recommend you to use data processing codes provided in GazeHub.

Official PyTorch implementation of
Official PyTorch implementation of "IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos", CVPRW 2021

IntegralAction: Pose-driven Feature Integration for Robust Human Action Recognition in Videos Introduction This repo is official PyTorch implementatio

Fashion Landmark Estimation with HRNet
Fashion Landmark Estimation with HRNet

HRNet for Fashion Landmark Estimation (Modified from deep-high-resolution-net.pytorch) Introduction This code applies the HRNet (Deep High-Resolution

[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.
[ECCV 2020] Reimplementation of 3DDFAv2, including face mesh, head pose, landmarks, and more.

Stable Head Pose Estimation and Landmark Regression via 3D Dense Face Reconstruction Reimplementation of (ECCV 2020) Towards Fast, Accurate and Stable

Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation
Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation

SimplePose Code and pre-trained models for our paper, “Simple Pose: Rethinking and Improving a Bottom-up Approach for Multi-Person Pose Estimation”, a

Owner
Sayom Shakib
Started with an electric motor and 2 AA battery once. And then once realized parallel circuit is just an OR gate! And that was all I needed.
Sayom Shakib
Code for BMVC2021 "MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation"

MOS-Multi-Task-Face-Detect Introduction This repo is the official implementation of "MOS: A Low Latency and Lightweight Framework for Face Detection,

null 100 Nov 21, 2022
Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

FAU Implementation of the paper: Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution. Yingruo

Evelyn 77 Nov 21, 2022
The official TensorFlow implementation of the paper Action Transformer: A Self-Attention Model for Short-Time Pose-Based Human Action Recognition

Action Transformer A Self-Attention Model for Short-Time Human Action Recognition This repository contains the official TensorFlow implementation of t

PIC4SeRCentre 13 Oct 3, 2022
Facial detection, landmark tracking and expression transfer library for Windows, Linux and Mac

Welcome to the CSIRO Face Analysis SDK. Documentation for the SDK can be found in doc/documentation.html. All code in this SDK is provided according t

Luiz Carlos Vieira 7 Jul 16, 2020
End-to-end face detection, cropping, norm estimation, and landmark detection in a single onnx model

onnx-facial-lmk-detector End-to-end face detection, cropping, norm estimation, and landmark detection in a single onnx model, model.onnx. Demo You can

atksh 41 Oct 27, 2022
Unit-Convertor - Unit Convertor Built With Python

Python Unit Converter This project can convert Weigth,length and ... units for y

Mahdis Esmaeelian 1 May 31, 2022
Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

Rafael Berral Soler 68 Nov 21, 2022
WHENet: Real-time Fine-Grained Estimation for Wide Range Head Pose

WHENet: Real-time Fine-Grained Estimation for Wide Range Head Pose Yijun Zhou and James Gregson - BMVC2020 Abstract: We present an end-to-end head-pos

null 362 Nov 26, 2022
Deep Learning Head Pose Estimation using PyTorch.

Hopenet is an accurate and easy to use head pose estimation network. Models have been trained on the 300W-LP dataset and have been tested on real data with good qualitative performance.

Nataniel Ruiz 1.3k Nov 29, 2022
Official Pytorch implementation of 6DRepNet: 6D Rotation representation for unconstrained head pose estimation.

6D Rotation Representation for Unconstrained Head Pose Estimation (Pytorch) Paper Thorsten Hempel and Ahmed A. Abdelrahman and Ayoub Al-Hamadi, "6D Ro

Thorsten Hempel 263 Nov 21, 2022