FaceAPI
AI-powered Face Detection & Rotation Tracking, Face Description & Recognition, Age & Gender & Emotion Prediction for Browser and NodeJS using TensorFlow/JS
Live Demo: https://vladmandic.github.io/face-api/demo/webcam.html
Additional Documentation
Examples
Browser
Browser example that uses static images and showcases both models
as well as all of the extensions is included in /demo/index.html
Example can be accessed directly using Git pages using URL:
https://vladmandic.github.io/face-api/demo/index.html
Browser example that uses live webcam is included in /demo/webcam.html
Example can be accessed directly using Git pages using URL:
https://vladmandic.github.io/face-api/demo/webcam.html
Demo using FaceAPI to process images
Note: Photos shown below are taken by me
Demo using FaceAPI to process live webcam
NodeJS
Three NodeJS examples are:
/demo/node.js
:
Regular usage ofFaceAPI
fromNodeJS
UsingTFJS
native methods to load images without external dependencies/demo/node-canvas.js
:
Regular usage ofFaceAPI
fromNodeJS
Using externalcanvas
module to load images
Which also allows for image drawing and saving insideNodeJS
environment/demo/node-multiprocess.js
:
Multiprocessing showcase that uses pool of worker processes
(node-multiprocess-worker.js
)
Main starts fixed pool of worker processes with each worker having
it's instance ofFaceAPI
Workers communicate with main when they are ready and main dispaches
job to each ready worker until job queue is empty
2021-03-14 08:42:03 INFO: @vladmandic/face-api version 1.0.2
2021-03-14 08:42:03 INFO: User: vlado Platform: linux Arch: x64 Node: v15.7.0
2021-03-14 08:42:03 INFO: FaceAPI multi-process test
2021-03-14 08:42:03 STATE: Main: started worker: 1888019
2021-03-14 08:42:03 STATE: Main: started worker: 1888025
2021-03-14 08:42:04 STATE: Worker: PID: 1888025 TensorFlow/JS 3.3.0 FaceAPI 1.0.2 Backend: tensorflow
2021-03-14 08:42:04 STATE: Worker: PID: 1888019 TensorFlow/JS 3.3.0 FaceAPI 1.0.2 Backend: tensorflow
2021-03-14 08:42:04 STATE: Main: dispatching to worker: 1888019
2021-03-14 08:42:04 STATE: Main: dispatching to worker: 1888025
2021-03-14 08:42:04 DATA: Worker received message: 1888019 { image: 'demo/sample1.jpg' }
2021-03-14 08:42:04 DATA: Worker received message: 1888025 { image: 'demo/sample2.jpg' }
2021-03-14 08:42:06 DATA: Main: worker finished: 1888025 detected faces: 3
2021-03-14 08:42:06 STATE: Main: dispatching to worker: 1888025
2021-03-14 08:42:06 DATA: Worker received message: 1888025 { image: 'demo/sample3.jpg' }
2021-03-14 08:42:06 DATA: Main: worker finished: 1888019 detected faces: 3
2021-03-14 08:42:06 STATE: Main: dispatching to worker: 1888019
2021-03-14 08:42:06 DATA: Worker received message: 1888019 { image: 'demo/sample4.jpg' }
2021-03-14 08:42:07 DATA: Main: worker finished: 1888025 detected faces: 3
2021-03-14 08:42:07 STATE: Main: dispatching to worker: 1888025
2021-03-14 08:42:07 DATA: Worker received message: 1888025 { image: 'demo/sample5.jpg' }
2021-03-14 08:42:08 DATA: Main: worker finished: 1888019 detected faces: 4
2021-03-14 08:42:08 STATE: Main: dispatching to worker: 1888019
2021-03-14 08:42:08 DATA: Worker received message: 1888019 { image: 'demo/sample6.jpg' }
2021-03-14 08:42:09 DATA: Main: worker finished: 1888025 detected faces: 5
2021-03-14 08:42:09 STATE: Main: worker exit: 1888025 0
2021-03-14 08:42:09 DATA: Main: worker finished: 1888019 detected faces: 4
2021-03-14 08:42:09 INFO: Processed 15 images in 5944 ms
2021-03-14 08:42:09 STATE: Main: worker exit: 1888019 0
Note that @tensorflow/tfjs-node
or @tensorflow/tfjs-node-gpu
must be installed before using NodeJS example
Quick Start
Simply include latest version of FaceAPI
directly from a CDN in your HTML:
(pick one, jsdelivr
or unpkg
)
<script src="https://cdn.jsdelivr.net/npm/@vladmandic/face-api/dist/face-api.js">script>
<script src="https://unpkg.dev/@vladmandic/face-api/dist/face-api.js">script>
Installation
FaceAPI
ships with several pre-build versions of the library:
dist/face-api.js
: IIFE format for client-side Browser execution
with TFJS pre-bundleddist/face-api.esm.js
: ESM format for client-side Browser execution
with TFJS pre-bundleddist/face-api.esm-nobundle.js
: ESM format for client-side Browser execution
without TFJS pre-bundleddist/face-api.node.js
: CommonJS format for server-side NodeJS execution
without TFJS pre-bundleddist/face-api.node-gpu.js
: CommonJS format for server-side NodeJS execution
without TFJS pre-bundled and optimized for CUDA GPU acceleration
Defaults are:
{
"main": "dist/face-api.node-js",
"module": "dist/face-api.esm.js",
"browser": "dist/face-api.esm.js",
}
Bundled TFJS
can be used directly via export: faceapi.tf
Reason for additional nobundle
version is if you want to
include a specific version of TFJS and not rely on pre-packaged one
FaceAPI
is compatible with TFJS 2.0+ and TFJS 3.0+
All versions include sourcemap
There are several ways to use FaceAPI:
1. IIFE script
Recommened for quick tests and backward compatibility with older Browsers that do not support ESM such as IE
This is simplest way for usage within Browser
Simply download dist/face-api.js
, include it in your HTML
file & it's ready to use:
<script src="https://cdn.jsdelivr.net/npm/@vladmandic/face-api/dist/face-api.js">script>
IIFE script bundles TFJS and auto-registers global namespace faceapi
within Window object which can be accessed directly from a tag or from your JS file.
2. ESM module
Recommended for usage within Browser
2.1. Direct Import
To use ESM import directly in a Browser, you must import your script (e.g. index.js
) with a type="module"
<script src="./index.js" type="module">
and then in your index.js
import * as faceapi from 'dist/face-api.esm.js';
2.2. With Bundler
Same as above, but expectation is that you've installed @vladmandic/faceapi
package:
npm install @vladmandic/face-api
and that you'll package your application using a bundler such as webpack
, rollup
or esbuild
in which case, you do not need to import a script as module - that depends on your bundler configuration
import * as faceapi from '@vladmandic/face-api';
or if your bundler doesn't recognize recommended
type, force usage with:
import * as faceapi from '@vladmandic/face-api/dist/face-api.esm.js';
or to use non-bundled version
import * as tf from `@tensorflow/tfjs`;
import * as faceapi from '@vladmandic/face-api/dist/face-api.esm-nobundle.js';
3. NPM module
3.1. Import CommonJS
Recommended for NodeJS projects
Node: FaceAPI for NodeJS does not bundle TFJS due to binary dependencies that are installed during TFJS installation
Install with:
npm install @tensorflow/tfjs-node
npm install @vladmandic/face-api
And then use with:
const tf = require('@tensorflow/tfjs-node')
const faceapi = require('@vladmandic/face-api');
If you want to force CommonJS module instead of relying on recommended
field:
const faceapi = require('@vladmandic/face-api/dist/face-api.node.js');
If you want to GPU Accelerated execution in NodeJS, you must have CUDA libraries already installed and working
Then install appropriate version of FaceAPI
:
npm install @tensorflow/tfjs-node
npm install @vladmandic/face-api
And then use with:
const tf = require('@tensorflow/tfjs-node-gpu')
const faceapi = require('@vladmandic/face-api/dist/face-api.node-gpu.js'); // this loads face-api version with correct bindings for tfjs-node-gpu
If you want to use FaceAPI
in a NodeJS on platforms where NodeJS binary libraries are not supported, you can use JavaScript CPU backend.
npm install @tensorflow/tfjs
npm install @vladmandic/face-api
And then use with:
const tf = require('@tensorflow/tfjs')
const faceapi = require('@vladmandic/face-api/dist/face-api.node-cpu.js');
If you want to use graphical functions inside NodeJS,
you must provide appropriate graphical library as
NodeJS does not include implementation for DOM elements
such as HTMLImageElement or HTMLCanvasElement:
Install Canvas
for NodeJS:
npm install canvas
Patch NodeJS environment to use newly installed Canvas
library:
const canvas = require('canvas');
const faceapi = require('@vladmandic/face-api');
const { Canvas, Image, ImageData } = canvas
faceapi.env.monkeyPatch({ Canvas, Image, ImageData })
Weights
Pretrained models and their weights are includes in ./model
.
Test & Dev Web Server
Built-in test&dev web server can be started using
npm run dev
By default it starts HTTP server on port 8000 and HTTPS server on port 8001 and can be accessed as:
2021-09-08 13:41:06 INFO: @vladmandic/face-api version 1.4.2
2021-09-08 13:41:06 INFO: User: vlado Platform: linux Arch: x64 Node: v16.8.0
2021-09-08 13:41:06 INFO: Application: { name: '@vladmandic/face-api', version: '1.4.2' }
2021-09-08 13:41:06 INFO: Environment: { profile: 'development', config: 'build.json', tsconfig: true, eslintrc: true, git: true }
2021-09-08 13:41:06 INFO: Toolchain: { esbuild: '0.12.25', typescript: '4.4.2', typedoc: '0.21.9', eslint: '7.32.0' }
2021-09-08 13:41:06 STATE: WebServer: { ssl: false, port: 8000, root: '.' }
2021-09-08 13:41:06 STATE: WebServer: { ssl: true, port: 8001, root: '.', sslKey: '/home/vlado/dev/build/cert/https.key', sslCrt: '/home/vlado/dev/build/cert/https.crt' }
2021-09-08 13:41:06 STATE: Watch: { locations: [ 'test/src/**', 'src/**' ] }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'cjs', platform: 'node', input: 'src/tfjs/tf-node.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 143, outputBytes: 1322 }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'cjs', platform: 'node', input: 'src/index.ts', output: 'dist/face-api.node.js', files: 162, inputBytes: 234423, outputBytes: 175260 }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'cjs', platform: 'node', input: 'src/tfjs/tf-node-gpu.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 147, outputBytes: 1330 }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'cjs', platform: 'node', input: 'src/index.ts', output: 'dist/face-api.node-gpu.js', files: 162, inputBytes: 234431, outputBytes: 175268 }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'cjs', platform: 'node', input: 'src/tfjs/tf-node-cpu.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 138, outputBytes: 1321 }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'cjs', platform: 'node', input: 'src/index.ts', output: 'dist/face-api.node-cpu.js', files: 162, inputBytes: 234422, outputBytes: 175259 }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'esm', platform: 'browser', input: 'src/tfjs/tf-browser.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 276, outputBytes: 272 }
2021-09-08 13:41:06 STATE: Build: { type: 'development', format: 'esm', platform: 'browser', input: 'src/index.ts', output: 'dist/face-api.esm-nobundle.js', files: 162, inputBytes: 233373, outputBytes: 169020 }
2021-09-08 13:41:07 STATE: Build: { type: 'development', format: 'esm', platform: 'browser', input: 'src/tfjs/tf-browser.ts', output: 'dist/tfjs.esm.js', files: 7, inputBytes: 276, outputBytes: 2371544 }
2021-09-08 13:41:07 STATE: Build: { type: 'development', format: 'iife', platform: 'browser', input: 'src/index.ts', output: 'dist/face-api.js', files: 162, inputBytes: 2604645, outputBytes: 2486481 }
2021-09-08 13:41:07 STATE: Build: { type: 'development', format: 'esm', platform: 'browser', input: 'src/index.ts', output: 'dist/face-api.esm.js', files: 162, inputBytes: 2604645, outputBytes: 2369658 }
.
.
2021-09-08 13:41:14 DATA: GET/2.0 200 text/html; charset=utf-8 1047 /demo/index.html ::1
2021-09-08 13:41:14 DATA: GET/2.0 200 text/javascript; charset=utf-8 6898 /demo/index.js ::1
2021-09-08 13:41:14 DATA: GET/2.0 200 text/javascript; charset=utf-8 2369658 /dist/face-api.esm.js ::1
2021-09-08 13:41:14 DATA: GET/2.0 200 application/octet-stream 6341199 /dist/face-api.esm.js.map ::1
2021-09-08 13:41:14 DATA: GET/2.0 200 application/json; charset=utf-8 3219 /model/tiny_face_detector_model-weights_manifest.json ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/octet-stream 193321 /model/tiny_face_detector_model.bin ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/json; charset=utf-8 28233 /model/ssd_mobilenetv1_model-weights_manifest.json ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/octet-stream 5616957 /model/ssd_mobilenetv1_model.bin ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/json; charset=utf-8 8392 /model/age_gender_model-weights_manifest.json ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/octet-stream 429708 /model/age_gender_model.bin ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/json; charset=utf-8 8485 /model/face_landmark_68_model-weights_manifest.json ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/octet-stream 356840 /model/face_landmark_68_model.bin ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/json; charset=utf-8 19615 /model/face_recognition_model-weights_manifest.json ::1
2021-09-08 13:41:15 DATA: GET/2.0 200 application/octet-stream 6444032 /model/face_recognition_model.bin ::1
2021-09-08 13:41:16 DATA: GET/2.0 200 application/json; charset=utf-8 6980 /model/face_expression_model-weights_manifest.json ::1
2021-09-08 13:41:16 DATA: GET/2.0 200 application/octet-stream 329468 /model/face_expression_model.bin ::1
2021-09-08 13:41:16 DATA: GET/2.0 200 image/jpg 144516 /demo/sample1.jpg ::1
Build
If you want to do a full rebuild, either download npm module
npm install @vladmandic/face-api
cd node_modules/@vladmandic/face-api
or clone a git project
git clone https://github.com/vladmandic/face-api
cd face-api
Then install all dependencies and run rebuild:
npm install
npm run build
Build process uses @vladmandic/build
module that creates optimized build for each target:
> @vladmandic/face-api@1.0.2 build
> rimraf dist/* types/* typedoc/* && node server/build.js
2021-09-08 13:40:19 INFO: @vladmandic/face-api version 1.4.2
2021-09-08 13:40:19 INFO: User: vlado Platform: linux Arch: x64 Node: v16.8.0
2021-09-08 13:40:19 INFO: Application: { name: '@vladmandic/face-api', version: '1.4.2' }
2021-09-08 13:40:19 INFO: Environment: { profile: 'production', config: 'build.json', tsconfig: true, eslintrc: true, git: true }
2021-09-08 13:40:19 INFO: Toolchain: { esbuild: '0.12.25', typescript: '4.4.2', typedoc: '0.21.9', eslint: '7.32.0' }
2021-09-08 13:40:19 STATE: Clean: { locations: [ 'dist/*', 'types/*', 'typedoc/*', [length]: 3 ] }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'cjs', platform: 'node', input: 'src/tfjs/tf-node.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 143, outputBytes: 1322 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'cjs', platform: 'node', input: 'src/index.ts', output: 'dist/face-api.node.js', files: 162, inputBytes: 234423, outputBytes: 175260 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'cjs', platform: 'node', input: 'src/tfjs/tf-node-gpu.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 147, outputBytes: 1330 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'cjs', platform: 'node', input: 'src/index.ts', output: 'dist/face-api.node-gpu.js', files: 162, inputBytes: 234431, outputBytes: 175268 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'cjs', platform: 'node', input: 'src/tfjs/tf-node-cpu.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 138, outputBytes: 1321 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'cjs', platform: 'node', input: 'src/index.ts', output: 'dist/face-api.node-cpu.js', files: 162, inputBytes: 234422, outputBytes: 175259 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'esm', platform: 'browser', input: 'src/tfjs/tf-browser.ts', output: 'dist/tfjs.esm.js', files: 1, inputBytes: 276, outputBytes: 272 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'esm', platform: 'browser', input: 'src/index.ts', output: 'dist/face-api.esm-nobundle.js', files: 162, inputBytes: 233373, outputBytes: 169020 }
2021-09-08 13:40:19 STATE: Build: { type: 'production', format: 'esm', platform: 'browser', input: 'src/tfjs/tf-browser.ts', output: 'dist/tfjs.esm.js', files: 7, inputBytes: 276, outputBytes: 2371544 }
2021-09-08 13:40:20 STATE: Build: { type: 'production', format: 'iife', platform: 'browser', input: 'src/index.ts', output: 'dist/face-api.js', files: 162, inputBytes: 2604645, outputBytes: 2486481 }
2021-09-08 13:40:20 STATE: Build: { type: 'production', format: 'esm', platform: 'browser', input: 'src/index.ts', output: 'dist/face-api.esm.js', files: 162, inputBytes: 2604645, outputBytes: 2369658 }
2021-09-08 13:40:22 STATE: Typings: { input: 'src/index.ts', output: 'types', files: 93 }
2021-09-08 13:40:27 STATE: TypeDoc: { input: 'src/index.ts', output: 'typedoc', objects: 154 }
2021-09-08 13:40:37 STATE: Lint: { locations: [ 'src/**', [length]: 1 ], files: 171, errors: 0, warnings: 0 }
2021-09-08 13:40:37 STATE: ChangeLog: { repository: 'https://github.com/vladmandic/face-api', branch: 'master', output: 'CHANGELOG.md' }
2021-09-08 13:40:37 INFO: Profile production done
Face Mesh
FaceAPI
landmark model returns 68-point face mesh as detailed in the image below:
Note
This is updated face-api.js with latest available TensorFlow/JS as the original is not compatible with tfjs 2.0+.
Forked from face-api.js version 0.22.2 which was released on March 22nd, 2020
Currently based on TensorFlow/JS
3.12.0
Why? I needed FaceAPI that does not cause version conflict with newer versions of TensorFlow
And since original FaceAPI was open-source, I've released this version as well
Changes ended up being too large for a simple pull request
and it ended up being a full-fledged version on its own
Plus many features were added since original inception
Although a lot of work has gone into this version of FaceAPI
and it will continue to be maintained,
at this time it is completely superseded by my newer library Human
which covers the same use cases,
but extends it with newer AI models, additional detection details, compatibility with latest web standard and more
Differences
Compared to face-api.js version 0.22.2:
- Compatible with
TensorFlow/JS 2.0+ & 3.0+
Originalface-api.js
is based onTFJS
1.7.4 - Compatible with
WebGL
,CPU
andWASM
TFJS Browser backends - Compatible with both
tfjs-node
andtfjs-node-gpu
TFJS NodeJS backends - Updated all type castings for TypeScript type checking to
TypeScript 4.4
- Switched bundling from
UMD
toESM
+CommonJS
with fallback toIIFE
Resulting code is optimized per-platform instead of being universal
Fully tree shakable when imported as anESM
module
Browser bundle process usesESBuild
instead ofRollup
- Added separate
face-api
versions withtfjs
pre-bundled and withouttfjs
When using-nobundle
version, user can load any version oftfjs
manually - Typescript build process now targets
ES2018
and instead of dualES5
/ES6
Resulting code is clean ES2018 JavaScript without polyfills - Removed old tests, docs, examples
- Removed old package dependencies (
karma
,jasmine
,babel
, etc.) - Updated all package dependencies
- Updated TensorFlow/JS dependencies since backends were removed from
@tensorflow/tfjs-core
- Updated
mobileNetv1
model due tobatchNorm()
dependency - Added
version
class that returns JSON object with version of FaceAPI as well as linked TFJS - Added test/dev built-in HTTP & HTTPS Web server
- Removed
mtcnn
andtinyYolov2
models as they were non-functional in latest public version ofFaceAPI
Which means valid models are tinyFaceDetector and mobileNetv1
If there is a demand, I can re-implement them back. - Added
face angle
calculations that returnsroll
,yaw
andpitch
- Added
typdoc
automatic API specification generation during build - Added
changelog
automatic generation during build
Credits
- Original project: face-api.js
- Original model weighs: face-api.js-models
- ML API Documentation: Tensorflow/JS