When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework (CVPR 2021 oral)

Overview

MTLFace

This repository contains the PyTorch implementation and the dataset of the paper: When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework (CVPR 2021 oral)

When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework
https://arxiv.org/abs/2103.01520
Abstract: To minimize the effects of age variation in face recognition, previous work either extracts identity-related discriminative features by minimizing the correlation between identity- and age-related features, called age-invariant face recognition (AIFR), or removes age variation by transforming the faces of different age groups into the same age group, called face age synthesis (FAS); however, the former lacks visual results for model interpretation while the latter suffers from artifacts compromising downstream recognition. Therefore, this paper proposes a unified, multi-task framework to jointly handle these two tasks, termed MTLFace, which can learn age-invariant identity-related representation while achieving pleasing face synthesis. Specifically, we first decompose the mixed face features into two uncorrelated components---identity- and age-related features---through an attention mechanism, and then decorrelate these two components using multi-task training and continuous domain adaption. In contrast to the conventional one-hot encoding that achieves group-level FAS, we propose a novel identity conditional module to achieve identity-level FAS, with a weight-sharing strategy to improve the age smoothness of synthesized faces. In addition, we collect and release a large cross-age face dataset with age and gender annotations to advance AIFR and FAS. Extensive experiments on five benchmark cross-age datasets demonstrate the superior performance of our proposed MTLFace over state-of-the-art methods for AIFR and FAS. We further validate MTLFace on two popular general face recognition datasets, showing competitive performance for face recognition in the wild.

example.png

framework

DATASET

  1. Downloading MS1M-ArcFace, CASIA-Webface or test set from insightface.
  2. Extract the jpg images from the mxnet .rec or .bin file according to the comments in the source code like:
python convert_insightface.py --source /home/zzhuang/faces_webface_112x112 --dest /home/zzhuang/casia-webface-112x112-arcface
python convert_insightface.py --bin --source /home/zzhuang/faces_webface_112x112/agedb_30.bin --dest /home/zzhuang/arcface-test-set
  1. Downloading the annotations from Dropbox, which is organized by id filename age gender; 1 for male and 0 for female.

  2. Putting the dataset and annotations into the dataset folder.

REQUIREMENTS

see requirements.txt and run pip install -r requirements.txt.

TRAINING

train AIFR:

python -m torch.distributed.launch --nproc_per_node=8 --master_port=17647 main.py \
    --train_fr --backbone_name ir50 --head_s 64 --head_m 0.35 \
    --weight_decay 5e-4 --momentum 0.9 --fr_age_loss_weight 0.001 --fr_da_loss_weight 0.002 --age_group 7 \
    --gamma 0.1 --milestone 20000 23000 --warmup 1000 --learning_rate 0.1 \
    --dataset_name scaf --image_size 112 --num_iter 36000 --batch_size 64 --amp

train FAS:

python -m torch.distributed.launch --nproc_per_node=8 --master_port=17647 main.py \
    --train_fas --backbone_name ir50 --age_group 7 \
    --dataset_name scaf --image_size 112 --num_iter 36000 --batch_size 64 \
    --d_lr 1e-4 --g_lr 1e-4 --fas_gan_loss_weight 75 --fas_age_loss_weight 10 --fas_id_loss_weight 0.002

If you want to train both tasks, please use apex.

Citation

If you found this code or our work useful please cite us:

@article{huang2020mtlface,
  title={When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework},
  author={Huang, Zhizhong and Zhang, Junping and Shan, Hongming},
  journal={CVPR},
  year={2021},
}

Acknowledgement

As my first CVPR paper, here I would appreciate all my co-authors and four anonymous reviewers for their valuable time, especially the one of them for his or her strong approvement to my work.

Comments
  • FAS visualization

    FAS visualization

    Thanks for your great job! I have tried to plot output of generator using torchvision.utils.make_grid but it still didn't work. This is my result: Screen Shot 2022-06-29 at 9 28 56 AM I am trying to make the result like this below image, can you give me the instruction ? Thanks ! Screen Shot 2022-06-29 at 9 33 47 AM

    opened by ngthalogit 2
  • Questions to Ask

    Questions to Ask

    Thank you for your answer, but there was another problem when I tried today. I thought about it for a long time but did not solve it. I come to ask you again, hoping to get your reply! image

    opened by liliiscaiji 1
  • train

    train

    Thank you very much for your contribution. May I ask if MTLFace will train AIFR model first and then FAS model? Or do I have to train at the same time? Looking forward to your reply!

    opened by liliiscaiji 1
  • some bug on AIFR training

    some bug on AIFR training

    Hi,when I was training AIFR,I found that the age prediction(get_dex_age) will tend to a fixed value like 35 or 32 years old after several iters.My dataset is casia-webface,and I didn't change other hyperparameters . Have you ever met this problem?Looking forward to your early reply!

    opened by sleepy678 1
  • Plot generated image from generator - FAS

    Plot generated image from generator - FAS

    Thanks for your great job!

    After train FAS, I try to plot the image generated from generator, but it does not work. Can you tell how to plot it correctly? Thanks

    opened by ngthalogit 1
  • Evaluation on AIFR

    Evaluation on AIFR

    Thank you for your contribution.

    I have a question about the evaluation on AIFR. You used a id-classification task fulfilling id-label prediction and calculating CE loss between id-pred-classification and id-gt on SCAF, while evaluated on AgeDB.

    I was wondering how can you evaluate on AgeDB30 since the id-gts are not given and obviously differ from SCAF, then how can you calculate the verification accuracy? Did you also train a verification module or just calculate similarity between "embedding" obtained by backbone of two inputs given by AgeDB30?

    Looking forward to your reply.

    Thanks

    opened by XinruiYuan 1
  • File not found error!!

    File not found error!!

    Hello, thanks for sharing your code. I am very interesting in your work. I run your code but meet this error:

    FileNotFoundError: File b'/data/xxx/Face/MTLFace/dataset/scaf.txt' does not exist

    The file is not in your baidu netdisk.

    opened by Watebear 1
  • regarding evaluation of the trained model

    regarding evaluation of the trained model

    Hi, Just wanted to ask that could you provide the code to run, test and evaluate the trained model and get the age progressions. The code to train the model is present but where can I find the code to test it?

    Thanks

    opened by ctsrkrk 1
  • FAS weights

    FAS weights

    Hi, I'm trying to visualize the result of FAS process, I have trained the casia-webface dataset with 36k iters following your instruction in readme file but it seems to be failed. Can you send me the file weights of FAS (generator.pt)? I just use it for visualization, pls help me ! Thanks. This is my gmail: [email protected]

    opened by ngthalogit 0
Owner
Hzzone
To be talented & positive.
Hzzone
Motion detector, Full body detection, Upper body detection, Cat face detection, Smile detection, Face detection (haar cascade), Silverware detection, Face detection (lbp), and Sending email notifications

Security camera running OpenCV for object and motion detection. The camera will send email with image of any objects it detects. It also runs a server that provides web interface with live stream video.

Peace 10 Jun 30, 2021
Official code for "Bridging Video-text Retrieval with Multiple Choice Questions", CVPR 2022 (Oral).

Bridging Video-text Retrieval with Multiple Choice Questions, CVPR 2022 (Oral) Paper | Project Page | Pre-trained Model | CLIP-Initialized Pre-trained

Applied Research Center (ARC), Tencent PCG 99 Jan 6, 2023
A curated list of resources for text detection/recognition (optical character recognition ) with deep learning methods.

awesome-deep-text-detection-recognition A curated list of awesome deep learning based papers on text detection and recognition. Text Detection Papers

null 2.4k Jan 8, 2023
Text recognition (optical character recognition) with deep learning methods.

What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis | paper | training and evaluation data | failure cases and cle

Clova AI Research 3.2k Jan 4, 2023
Sign Language Recognition service utilizing a deep learning model with Long Short-Term Memory to perform sign language recognition.

Sign Language Recognition Service This is a Sign Language Recognition service utilizing a deep learning model with Long Short-Term Memory to perform s

Martin Lønne 1 Jan 8, 2022
Polaris is a Face recognition attendance system .

Support Me ?? About Polaris ?? Polaris is a system based on facial recognition with a futuristic GUI design, Can easily find people informations store

XN3UR0N 215 Dec 26, 2022
EQFace: An implementation of EQFace: A Simple Explicit Quality Network for Face Recognition

EQFace: A Simple Explicit Quality Network for Face Recognition The first face recognition network that generates explicit face quality online.

DeepCam Shenzhen 141 Dec 31, 2022
Isearch (OSINT) 🔎 Face recognition reverse image search on Instagram profile feed photos.

isearch is an OSINT tool on Instagram. Offers a face recognition reverse image search on Instagram profile feed photos.

Malek salem 20 Oct 25, 2022
A facial recognition device is a device that takes an image or a video of a human face and compares it to another image faces in a database.

A facial recognition device is a device that takes an image or a video of a human face and compares it to another image faces in a database. The structure, shape and proportions of the faces are compared during the face recognition steps.

Pavankumar Khot 4 Mar 19, 2022
📷 Face Recognition using Haar-Cascade Classifier, OpenCV, and Python

Face-Recognition-System Face Recognition using Haar-Cascade Classifier, OpenCV and Python. This project is based on face detection and face recognitio

null 1 Jan 10, 2022
Opencv face recognition desktop application

Opencv-Face-Recognition Opencv face recognition desktop application Program developed by Gustavo Wydler Azuaga - 2021-11-19 Screenshots of the program

Gus 1 Nov 19, 2021
nofacedb/faceprocessor is a face recognition engine for NoFaceDB program complex.

faceprocessor nofacedb/faceprocessor is a face recognition engine for NoFaceDB program complex. Tech faceprocessor uses a number of open source projec

NoFaceDB 3 Sep 6, 2021
MORAN: A Multi-Object Rectified Attention Network for Scene Text Recognition

MORAN: A Multi-Object Rectified Attention Network for Scene Text Recognition Python 2.7 Python 3.6 MORAN is a network with rectification mechanism for

Canjie Luo 595 Dec 27, 2022
(CVPR 2021) Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds

BRNet Introduction This is a release of the code of our paper Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds,

null 86 Oct 5, 2022
Scale-aware Automatic Augmentation for Object Detection (CVPR 2021)

SA-AutoAug Scale-aware Automatic Augmentation for Object Detection Yukang Chen, Yanwei Li, Tao Kong, Lu Qi, Ruihang Chu, Lei Li, Jiaya Jia [Paper] [Bi

Jia Research Lab 182 Dec 29, 2022
(CVPR 2021) ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection

ST3D Code release for the paper ST3D: Self-training for Unsupervised Domain Adaptation on 3D Object Detection, CVPR 2021 Authors: Jihan Yang*, Shaoshu

CVMI Lab 224 Dec 28, 2022
Distilling Knowledge via Knowledge Review, CVPR 2021

ReviewKD Distilling Knowledge via Knowledge Review Pengguang Chen, Shu Liu, Hengshuang Zhao, Jiaya Jia This project provides an implementation for the

DV Lab 194 Dec 28, 2022
Code for the head detector (HeadHunter) proposed in our CVPR 2021 paper Tracking Pedestrian Heads in Dense Crowd.

Head Detector Code for the head detector (HeadHunter) proposed in our CVPR 2021 paper Tracking Pedestrian Heads in Dense Crowd. The head_detection mod

Ramana Subramanyam 76 Dec 6, 2022
Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

English | 简体中文 Introduction PaddleOCR aims to create multilingual, awesome, leading, and practical OCR tools that help users train better models and a

null 27.5k Jan 8, 2023