A Lightweight Face Recognition and Facial Attribute Analysis (Age, Gender, Emotion and Race) Library for Python

Overview

deepface

Downloads Stars License DOI

Deepface is a lightweight face recognition and facial attribute analysis (age, gender, emotion and race) framework for python. It is a hybrid face recognition framework wrapping state-of-the-art models: VGG-Face, Google FaceNet, OpenFace, Facebook DeepFace, DeepID, ArcFace and Dlib.

Experiments show that human beings have 97.53% accuracy on facial recognition tasks whereas those models already reached and passed that accuracy level.

Installation

The easiest way to install deepface is to download it from PyPI. It's going to install the library itself and its prerequisites as well. The library is mainly based on TensorFlow and Keras.

pip install deepface

Then you will be able to import the library and use its functionalities.

from deepface import DeepFace

Facial Recognition - Demo

A modern face recognition pipeline consists of 5 common stages: detect, align, normalize, represent and verify. Deepface handles all these common stages in the background. You can just call its verification, find or analysis function with a single line of code.

Face Verification - Demo

This function verifies face pairs as same person or different persons. It expects exact image paths as inputs. Passing numpy or based64 encoded images is also welcome.

result = DeepFace.verify(img1_path = "img1.jpg", img2_path = "img2.jpg")

Face recognition - Demo

Face recognition requires applying face verification many times. Herein, deepface has an out-of-the-box find function to handle this action. It's going to look for the identity of input image in the database path and it will return pandas data frame as output.

df = DeepFace.find(img_path = "img1.jpg", db_path = "C:/workspace/my_db")

Face recognition models - Demo

Deepface is a hybrid face recognition package. It currently wraps many state-of-the-art face recognition models: VGG-Face , Google FaceNet, OpenFace, Facebook DeepFace, DeepID, ArcFace and Dlib. The default configuration uses VGG-Face model.

models = ["VGG-Face", "Facenet", "Facenet512", "OpenFace", "DeepFace", "DeepID", "ArcFace", "Dlib"]
result = DeepFace.verify(img1_path = "img1.jpg", img2_path = "img2.jpg", model_name = models[1])
df = DeepFace.find(img_path = "img1.jpg", db_path = "C:/workspace/my_db", model_name = models[1])

FaceNet, VGG-Face, ArcFace and Dlib overperforms than OpenFace, DeepFace and DeepID based on experiments. Supportively, FaceNet /w 512d got 99.65%; FaceNet /w 128d got 99.2%; ArcFace got 99.41%; Dlib got 99.38%; VGG-Face got 98.78%; DeepID got 97.05; OpenFace got 93.80% accuracy scores on LFW data set whereas human beings could have just 97.53%.

Similarity

Face recognition models are regular convolutional neural networks and they are responsible to represent faces as vectors. We expect that a face pair of same person should be more similar than a face pair of different persons.

Similarity could be calculated by different metrics such as Cosine Similarity, Euclidean Distance and L2 form. The default configuration uses cosine similarity.

metrics = ["cosine", "euclidean", "euclidean_l2"]
result = DeepFace.verify(img1_path = "img1.jpg", img2_path = "img2.jpg", distance_metric = metrics[1])
df = DeepFace.find(img_path = "img1.jpg", db_path = "C:/workspace/my_db", distance_metric = metrics[1])

Euclidean L2 form seems to be more stable than cosine and regular Euclidean distance based on experiments.

Facial Attribute Analysis - Demo

Deepface also comes with a strong facial attribute analysis module including age, gender, facial expression (including angry, fear, neutral, sad, disgust, happy and surprise) and race (including asian, white, middle eastern, indian, latino and black) predictions.

obj = DeepFace.analyze(img_path = "img4.jpg", actions = ['age', 'gender', 'race', 'emotion'])

Age model got ± 4.65 MAE; gender model got 97.44% accuracy, 96.29% precision and 95.05% recall as mentioned in its tutorial.

Streaming and Real Time Analysis - Demo

You can run deepface for real time videos as well. Stream function will access your webcam and apply both face recognition and facial attribute analysis. The function starts to analyze a frame if it can focus a face sequantially 5 frames. Then, it shows results 5 seconds.

DeepFace.stream(db_path = "C:/User/Sefik/Desktop/database")

Even though face recognition is based on one-shot learning, you can use multiple face pictures of a person as well. You should rearrange your directory structure as illustrated below.

user
├── database
│   ├── Alice
│   │   ├── Alice1.jpg
│   │   ├── Alice2.jpg
│   ├── Bob
│   │   ├── Bob.jpg

Face Detectors - Demo

Face detection and alignment are early stages of a modern face recognition pipeline. Experiments show that just alignment increases the face recognition accuracy almost 1%. OpenCV, SSD, Dlib, MTCNN and RetinaFace detectors are wrapped in deepface. OpenCV is the default detector.

backends = ['opencv', 'ssd', 'dlib', 'mtcnn', 'retinaface']

#face detection and alignment
detected_face = DeepFace.detectFace(img_path = "img.jpg", detector_backend = backends[4])

#face verification
obj = DeepFace.verify(img1_path = "img1.jpg", img2_path = "img2.jpg", detector_backend = backends[4])

#face recognition
df = DeepFace.find(img_path = "img.jpg", db_path = "my_db", detector_backend = backends[4])

#facial analysis
demography = DeepFace.analyze(img_path = "img4.jpg", detector_backend = backends[4])

RetinaFace and MTCNN seem to overperform in detection and alignment stages but they are slower than others. If the speed of your pipeline is more important, then you should use opencv or ssd. On the other hand, if you consider the accuracy, then you should use retinaface or mtcnn.

API - Demo

Deepface serves an API as well. You can clone /api/api.py and pass it to python command as an argument. This will get a rest service up. In this way, you can call deepface from an external system such as mobile app or web.

python api.py

Face recognition, facial attribute analysis and vector representation functions are covered in the API. You are expected to call these functions as http post methods. Service endpoints will be http://127.0.0.1:5000/verify for face recognition, http://127.0.0.1:5000/analyze for facial attribute analysis, and http://127.0.0.1:5000/represent for vector representation. You should pass input images as base64 encoded string in this case. Here, you can find a postman project.

Tech Stack - Vlog, Tutorial

Face recognition models represent facial images as vector embeddings. The idea behind facial recognition is that vectors should be more similar for same person than different persons. The question is that where and how to store facial embeddings in a large scale system. Herein, deepface offers a represention function to find vector embeddings from facial images.

embedding = DeepFace.represent(img_path = "img.jpg", model_name = 'Facenet')

Tech stack is vast to store vector embeddings. To determine the right tool, you should consider your task such as face verification or face recognition, priority such as speed or confidence, and also data size.

Contribution

Pull requests are welcome. You should run the unit tests locally by running test/unit_tests.py. Please share the unit test result logs in the PR. Deepface is currently compatible with TF 1 and 2 versions. Change requests should satisfy those requirements both.

Support

There are many ways to support a project - starring ⭐️ the GitHub repo is just one.

Citation

Please cite deepface in your publications if it helps your research. Here are examples of its BibTeX entries:

@inproceedings{serengil2020lightface,
  title        = {LightFace: A Hybrid Deep Face Recognition Framework},
  author       = {Serengil, Sefik Ilkin and Ozpinar, Alper},
  booktitle    = {2020 Innovations in Intelligent Systems and Applications Conference (ASYU)},
  pages        = {23-27},
  year         = {2020},
  doi          = {10.1109/ASYU50717.2020.9259802},
  url          = {https://doi.org/10.1109/ASYU50717.2020.9259802},
  organization = {IEEE}
}
@inproceedings{serengil2021lightface,
  title        = {HyperExtended LightFace: A Facial Attribute Analysis Framework},
  author       = {Serengil, Sefik Ilkin and Ozpinar, Alper},
  booktitle    = {2021 International Conference on Engineering and Emerging Technologies (ICEET)},
  year         = {2021},
  organization = {IEEE}
}

Also, if you use deepface in your GitHub projects, please add deepface in the requirements.txt.

Licence

Deepface is licensed under the MIT License - see LICENSE for more details. However, the library wraps some external face recognition models: VGG-Face, Facenet, OpenFace, DeepFace, DeepID, ArcFace and Dlib. Besides, age, gender and race / ethnicity models are based on VGG-Face. Licence types will be inherited if you are going to use those models. Please check the license types of those models for production purposes.

Deepface logo is created by Adrien Coquet and it is licensed under Creative Commons: By Attribution 3.0 License.

Comments
  • [suggestion] Check face preprocessing step for all the keras face embedding models

    [suggestion] Check face preprocessing step for all the keras face embedding models

    In this repo, the face patch preprocessing step is same for all the models.

    img = cv2.resize(img, target_size)
    img_pixels = image.img_to_array(img)
    img_pixels = np.expand_dims(img_pixels, axis = 0)
    img_pixels /= 255 #normalize input in [0, 1]
    

    Please look into it if this is valid for all, as the inference preprocessing step must match the model's training pipeline for accurate result. For example, the original Facenet repo has done it differently.

    enhancement 
    opened by iamrishab 42
  • GPU Support

    GPU Support

    Hey i have nvidia 1070 GPU can you guide me to run this on gpu i Tried Deepface.allocateMemory() but it saying it will run on CPU. let me know the necessary tools as well as settings required to run this on GPU.

    Thanks

    opened by rockstarvibu 20
  • AttributeError: module 'keras.utils.generic_utils' has no attribute 'populate_dict_with_module_objects'

    AttributeError: module 'keras.utils.generic_utils' has no attribute 'populate_dict_with_module_objects'

    When using the deepface, I use conda the create a virsual envrionment, and using pip install deepface to install the packdge, but when I run the code, I met the error: AttributeError: module 'keras.utils.generic_utils' has no attribute 'populate_dict_with_module_objects' deepface==0.5.1 tensorflow==2.5.0 keras==2.4.3 image

    dependencies 
    opened by Hwang64 19
  • '_thread._local' object has no attribute 'value'

    '_thread._local' object has no attribute 'value'

    Hey there,

    I just started with DeepFace and it seemed all to work. But something weirds happen when I put the exact same code (copied from the official page) in a function.

    This is the code I try to run:

    ` try:

                faceinfo = DeepFace.analyze("images/happy.jpeg")
                print('[INFO] Emotion:', faceinfo["dominant_emotion"])
                print('[INFO] Age:', faceinfo["age"])
                print('[INFO] Gender:', faceinfo["gender"])
                print('[INFO] Race:', faceinfo["dominant_race"])
    
    
            except Exception as e:
                print(str(e) + ' [FACE INFO]')`
    

    The error i'm getting: '_thread._local' object has no attribute 'value'

    Could anyone help me out with this?

    opened by DevJoris 19
  • How to calculate candidate score in % in elastic search

    How to calculate candidate score in % in elastic search

    I'm using same code as you did in Elasticsearch. Im also using euclidean distance in query. I want to get score in % so I multiplied candidate_score * 100 but still results are always coming between 6% to 12%. can you tell me the process to get candidate score between 0 to 100%..

    Thanks

    question 
    opened by rockstarvibu 17
  • api is not support tensorflow version 2.2+

    api is not support tensorflow version 2.2+

    api.py not support tensorflow version 2.2.0+, when i run it, report error function get_default_graph not found, and it's can't load some models, report ValueError: rate must be a scalar tensor or a float in the range [0, 1) image

    bug 
    opened by gavinz8 17
  • E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303 on Docker Debian 9

    E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303 on Docker Debian 9

    I have this error using debian on the docker, I would like to disable cuda support, is there how?

    Below is the output:

    Using TensorFlow backend. ^[[AUsing VGG-Face model backend and cosine distance. 2020-05-20 14:35:39.426369: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303) Verification: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1.46it/s] {'pair_1': {'distance': 0.23476946353912354, 'max_threshold_to_verify': 0.4, 'model': 'VGG-Face', 'similarity_metric': 'cosine', 'verified': True}}

    opened by dan-developer 17
  • appending functionality to .pkl files in deepface find

    appending functionality to .pkl files in deepface find

    Dear Mr.Sefik! I like the combination between facenet and mtcnn for face recognition, not analyze But I have 2 troubles: 1/ When I execute DeepFace.find, this function generates a file in folder (representations_facenet.pkl), I want to append for new value, new image, I failed because I don't know format statement in file
    df = pd.DataFrame(representations, columns = ["identity", "%s_representation" % (model_name)]). How to do that? I tried to write file [a+] or [wb] and that thing, it doesn't work with next time detection | recognition 2/ the reason I do the number 1 thing is - Loading speed for 1 vector file takes a lot of seconds each time ~43 seconds of Facenet model, I agree that the second time is very fast, only 8 seconds [time import, time find function], but the recognition is not correct. Finally, I have to delete the old pkl file and do again. There is no way to improve adding new to the pkl . file. I want to add or combine file, and speed more faster, about 3-5 seconds.

     Thats all | Help me, thanks
    
    enhancement question 
    opened by LOANPIA 15
  • running deepface on gpu

    running deepface on gpu

    I have installed tensorflow-gpu instead of tensortflow after doing pip install deepface. How can I run deepface on a gpu? or be sure that deepface is running on a GPU? DeepFace.verify with 'VGG-Face' takes close to 6 seconds. I would like to reduce this by using a GPU.

    question 
    opened by ashishlal 15
  • Process finished with exit code 137 (interrupted by signal 9: SIGKILL)

    Process finished with exit code 137 (interrupted by signal 9: SIGKILL)

    I am doing experiment on large scale datasets with extensive memory and GPU. Actually, when i started an experiments on Ensemble-Face-Recognition my process has been killed. Is it possible to consume my GPU for these experiments.

    (236167, 3)
    Process finished with exit code 137 (interrupted by signal 9: SIGKILL)
    
    question 
    opened by khawar-islam 14
  • Issue with Ensemble model.

    Issue with Ensemble model.

    When I want to use Ensemble model, like the tutorial"

    resp_obj = DeepFace.verify("img1.jpg", "img2.jpg", model_name = "Ensemble")

    I get the following error: There is an error with this image and modelCould not open /usr/local/lib/python3.6/dist-packages/deepface/models/face-recognition-ensemble-model.txt

    How we can solve that?

    the library version is 0.0.31

    bug 
    opened by MahsaSeifikar 14
  • Add support for `png` and `webp` images in local database

    Add support for `png` and `webp` images in local database

    Description

    This PR adds the ability to use png, webp and jpg|jpeg images in local image database.

    The commons module now includes an utils.py file that contains:

    • A list of default supported image extensions
    • Helper to extract the file extension from a given filename
    • Helper to check if a given filename ends with any of the supported file extensions
    • Helper to generate a label for a canidate from a given path

    Fixes #612

    Type of change

    • [x] New feature (non-breaking change which adds functionality)
    • [x] This change requires a documentation update

    Current behavior

    • Only allows images in jpg format

    New behavior

    • In addition to the current behavior, also allows using png and webp images
    • Supports both JPEG extensions - jpeg | jpg

    Tests

    • [x] Simple tests included in test/utils.py
    • [x] Run tests/unit_test.py
    opened by ekkolon 0
  • load_image is not same from url or from file

    load_image is not same from url or from file

    # in colab
    from deepface.commons import functions
    import numpy as np
    url = "https://upload.wikimedia.org/wikipedia/commons/thumb/4/45/MontreGousset001.jpg/440px-MontreGousset001.jpg"
    file = "440px-MontreGousset001.jpg"
    !wget {url} -O {file} -q
    
    img_from_url = functions.load_image(url)
    img_from_file = functions.load_image(file)
    
    print(np.abs(img_from_url-img_from_file).mean())
    # 98.4184350027692
    

    https://github.com/serengil/deepface/blob/ba9f56755abdc9d1d80777d7680c4267d5d5e0e9/deepface/commons/functions.py#L85-L92 The reason is because you convert the image to RBG with url And for file, you use cv2.imread, the default setting is rgb

    Please align those two methods

    dependencies 
    opened by mistake0316 0
  • added option to detect multiple faces in detectFace

    added option to detect multiple faces in detectFace

    Hi! I wanted to detect multiple faces in an image but the library didn't offer any support for multiple face detection. Detecting multiple faces in the images might be a use case for other developers too!

    I've run unit_test.py and attached the output : unit_tests.txt

    PS: I am not a pro. Please suggest any design changes if it can be done in a better way!

    opened by iakshat 3
Owner
Sefik Ilkin Serengil
👨‍💻Software Engineer 🎓GSU alumni ⌨️Blogger 🏠Istanbulite 💬Code wins arguments
Sefik Ilkin Serengil
Face Detection & Age Gender & Expression & Recognition

Face Detection & Age Gender & Expression & Recognition

Sajjad Ayobi 188 Dec 28, 2022
FACIAL: Synthesizing Dynamic Talking Face With Implicit Attribute Learning. ICCV, 2021.

FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learning PyTorch implementation for the paper: FACIAL: Synthesizing Dynamic Talking

null 226 Jan 8, 2023
This is an easy python software which allows to sort images with faces by gender and after by age.

Gender-age Classifier This is an easy python software which allows to sort images with faces by gender and after by age. Usage First install Deepface

Claudio Ciccarone 6 Sep 17, 2022
Python implementation of a live deep learning based age/gender/expression recognizer

TUT live age estimator Python implementation of a live deep learning based age/gender/smile/celebrity twin recognizer. All components use convolutiona

Heikki Huttunen 80 Nov 21, 2022
A deep learning network built with TensorFlow and Keras to classify gender and estimate age.

Convolutional Neural Network (CNN). This repository contains a source code of a deep learning network built with TensorFlow and Keras to classify gend

Pawel Dziemiach 1 Dec 18, 2021
A deep learning network built with TensorFlow and Keras to classify gender and estimate age.

Convolutional Neural Network (CNN). This repository contains a source code of a deep learning network built with TensorFlow and Keras to classify gend

Pawel Dziemiach 1 Dec 19, 2021
Age and Gender prediction using Keras

cnn_age_gender Age and Gender prediction using Keras Dataset example : Description : UTKFace dataset is a large-scale face dataset with long age span

XN3UR0N 58 May 3, 2022
This is a Keras implementation of a CNN for estimating age, gender and mask from a camera.

face-detector-age-gender This is a Keras implementation of a CNN for estimating age, gender and mask from a camera. Before run face detector app, expr

Devdreamsolution 2 Dec 4, 2021
SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model

SEOVER-Master This code is the implementation of paper: SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model

null 4 Feb 24, 2022
Method for facial emotion recognition compitition of Xunfei and Datawhale .

人脸情绪识别挑战赛-第3名-W03KFgNOc-源代码、模型以及说明文档 队名:W03KFgNOc 排名:3 正确率: 0.75564 队员:yyMoming,xkwang,RichardoMu。 比赛链接:人脸情绪识别挑战赛 文章地址:link emotion 该项目分别训练八个模型并生成csv文

null 6 Oct 17, 2022
Testing the Facial Emotion Recognition (FER) algorithm on animations

PegHeads-Tutorial-3 Testing the Facial Emotion Recognition (FER) algorithm on animations

PegHeads Inc 2 Jan 3, 2022
Emotion Recognition from Facial Images

Reconhecimento de Emoções a partir de imagens faciais Este projeto implementa um classificador simples que utiliza técncias de deep learning e transfe

Gabriel 2 Feb 9, 2022
Web service for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation based on OpenFace 2.0

OpenGaze: Web Service for OpenFace Facial Behaviour Analysis Toolkit Overview OpenFace is a fantastic tool intended for computer vision and machine le

Sayom Shakib 4 Nov 3, 2022
OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.

OpenFace 2.2.0: a facial behavior analysis toolkit Over the past few years, there has been an increased interest in automatic facial behavior analysis

Tadas Baltrusaitis 5.8k Dec 31, 2022
An implementation of paper `Real-time Convolutional Neural Networks for Emotion and Gender Classification` with PaddlePaddle.

简介 通过PaddlePaddle框架复现了论文 Real-time Convolutional Neural Networks for Emotion and Gender Classification 中提出的两个模型,分别是SimpleCNN和MiniXception。利用 imdb_crop

null 8 Mar 11, 2022
Automatically measure the facial Width-To-Height ratio and get facial analysis results provided by Microsoft Azure

fwhr-calc-website This project is to automatically measure the facial Width-To-Height ratio and get facial analysis results provided by Microsoft Azur

SoohyunPark 1 Feb 7, 2022
Face-Recognition-Attendence-System - This face recognition Attendence system using Python

Face-Recognition-Attendence-System I have developed this face recognition Attend

Riya Gupta 4 May 10, 2022
Face Recognition and Emotion Detector Device

Face Recognition and Emotion Detector Device Orange PI 1 Python 3.10.0 + Django 3.2.9 Project's file explanation Django manage.py Django commands hand

BootyAss 2 Dec 21, 2021
Face Library is an open source package for accurate and real-time face detection and recognition

Face Library Face Library is an open source package for accurate and real-time face detection and recognition. The package is built over OpenCV and us

null 52 Nov 9, 2022