MOS-Multi-Task-Face-Detect
Introduction
This repo is the official implementation of "MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation". The paper has been accepted at BMVC2021.
This repo is an implementation of PyTorch. MOS is a low latency and lightweight architecture for face detection, facial landmark localization and head pose estimation.It aims to bridge the gap between research and industrial communities. For more details, please refer to our report on Arxiv.
Updates
- 【2021/10/31】 We have released the training data (widerface with pose label). The pytorch inference code of MOS-S and MOS-M has been released!
- 【2021/10/22】 We have released our paper on Arxiv.
- 【2021/10/15】 "MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation" has been accepted at BMVC2021.
Comming soon
- Tensorrt inference code.
- Openvino inference code.
- Ncnn inference code.
- The fastest version: MOS-tiny.
Benchmark
Light Models.
WiderFace Val Performance is in multi scale and Pose evaluation is using AFLW2000 in 300X300 as image input.
| Model | backbone | easy | medium | hard | pitch | yaw | roll |
|---|---|---|---|---|---|---|---|
| MOS-M | mobilenetV2 | 94.08 | 93.21 | 88.06 | 6.67 | 4.43 | 5.83 |
| MOS-S | shufflenetV2 | 93.28 | 92.12 | 86.97 | 6.80 | 4.28 | 5.99 |
generate widerface validation results
- Generate txt file You need download the validation and test dataset of WiderFace from Here
python test_widerface.py --network cfg_mos_m --trained_model ./test_weights/MOS-M.pth
- Evaluate txt results. Demo come from Here
cd ./widerface_evaluate
python setup.py build_ext --inplace
python evaluation.py
Training data
- Download annotations (face bounding boxes & five facial landmarks & pose angle(pitch,yaw,roll)) from baidu cloud , the code is 0925. We also provide the GOOGLE DRIVE
- Organise the dataset directory as follows:
./data/widerface/
train/
images/
label.txt
The annotation file is like:
# 0--Parade/0_Parade_marchingband_1_849.jpg
449 330 122 149 488.906 373.643 0.0 542.089 376.442 0.0 515.031 412.83 0.0 485.174 425.893 0.0 538.357 431.491 0.0 0.82 -6 -6 1
face_x face_y face_width face_height landmark1.x landmark1.y 0.0 landmark2.x landmark2.y 0.0 landmark3.x landmark3.y 0.0 landmark4.x landmark4.y 0.0
landmark5.x landmark5.y 0.0 confidence pitch yaw roll
Quick Start
Step1. Install MOS.
git clone https://github.com/lyp-deeplearning/MOS-Multi-Task-Face-Detect.git
cd MOS-Multi-Task-Face-Detect
conda create -n MOS python=3.8.5
conda activate MOS
pip install -r requirements.txt
cd models/DCNv2/
python setup.py build develop
Step2. Run Pytorch inference demo.
## run the MOS-M model
python detect_picture.py --network cfg_mos_m --trained_model ./test_weights/MOS-M.pth
## run the MOS-S model
python detect_picture.py --network cfg_mos_s --trained_model ./test_weights/MOS-S.pth
Step3. Run video inference demo.
## run the MOS-M model
python detect_video.py --network cfg_mos_m --trained_model ./test_weights/MOS-M.pth
Cite MOS
If you use MOS in your research, please cite our work by using the following BibTeX entry:
@article{liu2021mos,
title={MOS: A Low Latency and Lightweight Framework for Face Detection, Landmark Localization, and Head Pose Estimation},
author={Liu, Yepeng and Gu, Zaiwang and Gao, Shenghua and Wang, Dong and Zeng, Yusheng and Cheng, Jun},
journal={arXiv preprint arXiv:2110.10953},
year={2021}
}
I got above error when compiling DCNv2 using setup.py. My device is my personal PC (win11,AMD64,x64-based PC). My torch version is 1.11.0+cu113. i don't know whether i should update DCNv2 to corresponding version, and such problem can be solve?
Could you please give any help?
4 Dec 19, 2021
11 Oct 21, 2021
17 Dec 20, 2022
3 Oct 22, 2021
66 Dec 26, 2022
73 Nov 6, 2022
18 Oct 6, 2022
94 Jan 6, 2023
72 Jan 6, 2023
3 Jul 14, 2021
56 Nov 15, 2022
408 Jan 1, 2023
1 Oct 26, 2021
7 Apr 5, 2022
4 May 26, 2022
32 Nov 9, 2021
16 Oct 11, 2022
3k Jan 5, 2023
1k Jan 2, 2023