InsightFace: 2D and 3D Face Analysis Project
By Jia Guo and Jiankang Deng
Top News
2021-06-05: We launch a Masked Face Recognition Challenge & Workshop on ICCV 2021.
2021-05-15: We released an efficient high accuracy face detection approach called SCRFD.
2021-04-18: We achieved Rank-4th on NIST-FRVT 1:1, see leaderboard.
2021-03-13: We have released our official ArcFace PyTorch implementation, see here.
License
The code of InsightFace is released under the MIT License. There is no limitation for both academic and commercial usage.
The training data containing the annotation (and the models trained with these data) are available for non-commercial research purposes only.
Introduction
InsightFace is an open source 2D&3D deep face analysis toolbox, mainly based on MXNet and PyTorch.
The master branch works with MXNet 1.2 to 1.6, PyTorch 1.6+, with Python 3.x.
ArcFace Video Demo
Please click the image to watch the Youtube video. For Bilibili users, click here.
Recent Update
2021-06-05: We launch a Masked Face Recognition Challenge & Workshop on ICCV 2021.
2021-05-15: We released an efficient high accuracy face detection approach called SCRFD.
2021-04-18: We achieved Rank-4th on NIST-FRVT 1:1, see leaderboard.
2021-03-13: We have released our official ArcFace PyTorch implementation, see here.
2021-03-09: Tips for training large-scale face recognition model, such as millions of IDs(classes).
2021-02-21: We provide a simple face mask renderer here which can be used as a data augmentation tool while training face recognition models.
2021-01-20: OneFlow based implementation of ArcFace and Partial-FC, here.
2020-10-13: A new training method and one large training set(360K IDs) were released here by DeepGlint.
2020-10-09: We opened a large scale recognition test benchmark IFRT
2020-08-01: We released lightweight facial landmark models with fast coordinate regression(106 points). See detail here.
2020-04-27: InsightFace pretrained models and MS1M-Arcface are now specified as the only external training dataset, for iQIYI iCartoonFace challenge, see detail here.
2020.02.21: Instant discussion group created on QQ with group-id: 711302608. For English developers, see install tutorial here.
2020.02.16: RetinaFace now can detect faces with mask, for anti-CoVID19, see detail here
2019.08.10: We achieved 2nd place at WIDER Face Detection Challenge 2019.
2019.05.30: Presentation at cvmart
2019.04.30: Our Face detector (RetinaFace) obtains state-of-the-art results on the WiderFace dataset.
2019.04.14: We will launch a Light-weight Face Recognition challenge/workshop on ICCV 2019.
2019.04.04: Arcface achieved state-of-the-art performance (7/109) on the NIST Face Recognition Vendor Test (FRVT) (1:1 verification) report (name: Imperial-000 and Imperial-001). Our solution is based on [MS1MV2+DeepGlintAsian, ResNet100, ArcFace loss].
2019.02.08: Please check https://github.com/deepinsight/insightface/tree/master/recognition/ArcFace for our parallel training code which can easily and efficiently support one million identities on a single machine (8* 1080ti).
2018.12.13: Inference acceleration TVM-Benchmark.
2018.10.28: Light-weight attribute model Gender-Age. About 1MB, 10ms on single CPU core. Gender accuracy 96% on validation set and 4.1 age MAE.
2018.10.16: We achieved state-of-the-art performance on Trillionpairs (name: nttstar) and IQIYI_VID (name: WitcheR).
Contents
- Introduction
- Training Data
- Train
- Pretrained Models
- Verification Results On Combined Margin
- Test on MegaFace
- 512-D Feature Embedding
- Third-party Re-implementation
Deep Face Recognition
Introduction
In this module, we provide training data, network settings and loss designs for deep face recognition. The training data includes, but not limited to the cleaned MS1M, VGG2 and CASIA-Webface datasets, which were already packed in MXNet binary format. The network backbones include ResNet, MobilefaceNet, MobileNet, InceptionResNet_v2, DenseNet, etc.. The loss functions include Softmax, SphereFace, CosineFace, ArcFace, Sub-Center ArcFace and Triplet (Euclidean/Angular) Loss.
You can check the detail page of our work ArcFace(which accepted in CVPR-2019) and SubCenter-ArcFace(which accepted in ECCV-2020).
Our method, ArcFace, was initially described in an arXiv technical report. By using this module, you can simply achieve LFW 99.83%+ and Megaface 98%+ by a single model. This module can help researcher/engineer to develop deep face recognition algorithms quickly by only two steps: download the binary dataset and run the training script.
Training Data
All face images are aligned by ficial five landmarks and cropped to 112x112:
Please check Dataset-Zoo for detail information and dataset downloading.
- Please check recognition/tools/face2rec2.py on how to build a binary face dataset. You can either choose MTCNN or RetinaFace to align the faces.
Train
- Install
MXNetwith GPU support (Python 3.X).
pip install mxnet-cu101 # which should match your installed cuda version
- Clone the InsightFace repository. We call the directory insightface as
INSIGHTFACE_ROOT.
git clone --recursive https://github.com/deepinsight/insightface.git
- Download the training set (
MS1M-Arcface) and place it in$INSIGHTFACE_ROOT/recognition/datasets/. Each training dataset includes at least following 6 files:
faces_emore/
train.idx
train.rec
property
lfw.bin
cfp_fp.bin
agedb_30.bin
The first three files are the training dataset while the last three files are verification sets.
- Train deep face recognition models. In this part, we assume you are in the directory
$INSIGHTFACE_ROOT/recognition/ArcFace/.
Place and edit config file:
cp sample_config.py config.py
vim config.py # edit dataset path etc..
We give some examples below. Our experiments were conducted on the Tesla P40 GPU.
(1). Train ArcFace with LResNet100E-IR.
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train.py --network r100 --loss arcface --dataset emore
It will output verification results of LFW, CFP-FP and AgeDB-30 every 2000 batches. You can check all options in config.py. This model can achieve LFW 99.83+ and MegaFace 98.3%+.
(2). Train CosineFace with LResNet50E-IR.
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train.py --network r50 --loss cosface --dataset emore
(3). Train Softmax with LMobileNet-GAP.
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train.py --network m1 --loss softmax --dataset emore
(4). Fine-turn the above Softmax model with Triplet loss.
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train.py --network m1 --loss triplet --lr 0.005 --pretrained ./models/m1-softmax-emore,1
(5). Training in model parallel acceleration.
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train_parall.py --network r100 --loss arcface --dataset emore
- Verification results.
LResNet100E-IR network trained on MS1M-Arcface dataset with ArcFace loss:
| Method | LFW(%) | CFP-FP(%) | AgeDB-30(%) |
|---|---|---|---|
| Ours | 99.80+ | 98.0+ | 98.20+ |
Pretrained Models
You can use $INSIGHTFACE_ROOT/recognition/arcface_torch/eval/verification.py to test all the pre-trained models.
Please check Model-Zoo for more pretrained models.
Verification Results on Combined Margin
A combined margin method was proposed as a function of target logits value and original θ:
COM(θ) = cos(m_1*θ+m_2) - m_3
For training with m1=1.0, m2=0.3, m3=0.2, run following command:
CUDA_VISIBLE_DEVICES='0,1,2,3' python -u train.py --network r100 --loss combined --dataset emore
Results by using MS1M-IBUG(MS1M-V1)
| Method | m1 | m2 | m3 | LFW | CFP-FP | AgeDB-30 |
|---|---|---|---|---|---|---|
| W&F Norm Softmax | 1 | 0 | 0 | 99.28 | 88.50 | 95.13 |
| SphereFace | 1.5 | 0 | 0 | 99.76 | 94.17 | 97.30 |
| CosineFace | 1 | 0 | 0.35 | 99.80 | 94.4 | 97.91 |
| ArcFace | 1 | 0.5 | 0 | 99.83 | 94.04 | 98.08 |
| Combined Margin | 1.2 | 0.4 | 0 | 99.80 | 94.08 | 98.05 |
| Combined Margin | 1.1 | 0 | 0.35 | 99.81 | 94.50 | 98.08 |
| Combined Margin | 1 | 0.3 | 0.2 | 99.83 | 94.51 | 98.13 |
| Combined Margin | 0.9 | 0.4 | 0.15 | 99.83 | 94.20 | 98.16 |
Test on MegaFace
Please check $INSIGHTFACE_ROOT/evaluation/megaface/ to evaluate the model accuracy on Megaface. All aligned images were already provided.
512-D Feature Embedding
In this part, we assume you are in the directory $INSIGHTFACE_ROOT/deploy/. The input face image should be generally centre cropped. We use RNet+ONet of MTCNN to further align the image before sending it to the feature embedding network.
- Prepare a pre-trained model.
- Put the model under
$INSIGHTFACE_ROOT/models/. For example,$INSIGHTFACE_ROOT/models/model-r100-ii. - Run the test script
$INSIGHTFACE_ROOT/deploy/test.py.
For single cropped face image(112x112), total inference time is only 17ms on our testing server(Intel E5-2660 @ 2.00GHz, Tesla M40, LResNet34E-IR).
Third-party Re-implementation
- TensorFlow: InsightFace_TF
- TensorFlow: tf-insightface
- TensorFlow:insightface
- PyTorch: InsightFace_Pytorch
- PyTorch: arcface-pytorch
- Caffe: arcface-caffe
- Caffe: CombinedMargin-caffe
- Tensorflow: InsightFace-tensorflow
- TensorRT: wang-xinyu/tensorrtx
Face Detection
RetinaFace
RetinaFace is a practical single-stage SOTA face detector which is initially introduced in arXiv technical report and then accepted by CVPR 2020. We provide training code, training dataset, pretrained models and evaluation scripts.
Please check RetinaFace for detail.
RetinaFaceAntiCov
RetinaFaceAntiCov is an experimental module to identify face boxes with masks. Please check RetinaFaceAntiCov for detail.
Face Alignment
DenseUNet
Please check the Menpo Benchmark and our Dense U-Net for detail. We also provide other network settings such as classic hourglass. You can find all of training code, training dataset and evaluation scripts there.
CoordinateReg
On the other hand, in contrast to heatmap based approaches, we provide some lightweight facial landmark models with fast coordinate regression. The input of these models is loose cropped face image while the output is the direct landmark coordinates. See detail at alignment-coordinateReg. Now only pretrained models available.
Citation
If you find InsightFace useful in your research, please consider to cite the following related papers:
@inproceedings{deng2019retinaface,
title={RetinaFace: Single-stage Dense Face Localisation in the Wild},
author={Deng, Jiankang and Guo, Jia and Yuxiang, Zhou and Jinke Yu and Irene Kotsia and Zafeiriou, Stefanos},
booktitle={arxiv},
year={2019}
}
@inproceedings{guo2018stacked,
title={Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment},
author={Guo, Jia and Deng, Jiankang and Xue, Niannan and Zafeiriou, Stefanos},
booktitle={BMVC},
year={2018}
}
@article{deng2018menpo,
title={The Menpo benchmark for multi-pose 2D and 3D facial landmark localisation and tracking},
author={Deng, Jiankang and Roussos, Anastasios and Chrysos, Grigorios and Ververas, Evangelos and Kotsia, Irene and Shen, Jie and Zafeiriou, Stefanos},
journal={IJCV},
year={2018}
}
@inproceedings{deng2018arcface,
title={ArcFace: Additive Angular Margin Loss for Deep Face Recognition},
author={Deng, Jiankang and Guo, Jia and Niannan, Xue and Zafeiriou, Stefanos},
booktitle={CVPR},
year={2019}
}
Contact
[Jia Guo](guojia[at]gmail.com)
[Jiankang Deng](jiankangdeng[at]gmail.com)
7.5k Dec 28, 2022
31 Oct 19, 2019
100 Dec 10, 2022
549 Dec 28, 2022
531 Dec 4, 2022
52 Nov 9, 2022
38 Oct 21, 2022
519 Dec 29, 2022
99 Dec 30, 2022
86 Aug 3, 2022
1.7k Dec 26, 2022
77 Dec 8, 2022
52 Dec 30, 2022
17 Apr 26, 2022
608 Jan 2, 2023
232 Dec 29, 2022
2.3k Dec 30, 2022
4 May 10, 2022
1 Jan 13, 2022