🔥🔥High-Performance Face Recognition Library on PaddlePaddle & PyTorch🔥🔥

Overview

face.evoLVe: High-Performance Face Recognition Library based on PaddlePaddle & PyTorch

  • Evolve to be more comprehensive, effective and efficient for face related analytics & applications! (WeChat News)
  • About the name:
    • "face" means this repo is dedicated for face related analytics & applications.
    • "evolve" means unleash your greatness to be better and better. "LV" are capitalized to acknowledge the nurturing of Learning and Vision (LV) group, Nation University of Singapore (NUS).
  • This work was done during Jian Zhao served as a short-term "Texpert" Research Scientist at Tencent FiT DeepSea AI Lab, Shenzhen, China.
Author Jian Zhao
Homepage https://zhaoj9014.github.io

License

The code of face.evoLVe is released under the MIT License.


News

CLOSED 02 September 2021: Baidu PaddlePaddle officially merged face.evoLVe to faciliate researches and applications on face-related analytics (Official Announcement).

CLOSED 03 July 2021: Provides training code for the paddlepaddle framework.

CLOSED 04 July 2019: We will share several publicly available datasets on face anti-spoofing/liveness detection to facilitate related research and analytics.

CLOSED 07 June 2019: We are training a better-performing IR-152 model on MS-Celeb-1M_Align_112x112, and will release the model soon.

CLOSED 23 May 2019: We share three publicly available datasets to facilitate research on heterogeneous face recognition and analytics. Please refer to Sec. Data Zoo for details.

CLOSED 23 Jan 2019: We share the name lists and pair-wise overlapping lists of several widely-used face recognition datasets to help researchers/engineers quickly remove the overlapping parts between their own private datasets and the public datasets. Please refer to Sec. Data Zoo for details.

CLOSED 23 Jan 2019: The current distributed training schema with multi-GPUs under PyTorch and other mainstream platforms parallels the backbone across multi-GPUs while relying on a single master to compute the final bottleneck (fully-connected/softmax) layer. This is not an issue for conventional face recognition with moderate number of identities. However, it struggles with large-scale face recognition, which requires recognizing millions of identities in the real world. The master can hardly hold the oversized final layer while the slaves still have redundant computation resource, leading to small-batch training or even failed training. To address this problem, we are developing a highly-elegant, effective and efficient distributed training schema with multi-GPUs under PyTorch, supporting not only the backbone, but also the head with the fully-connected (softmax) layer, to facilitate high-performance large-scale face recognition. We will added this support into our repo.

CLOSED 22 Jan 2019: We have released two feature extraction APIs for extracting features from pre-trained models, implemented with PyTorch build-in functions and OpenCV, respectively. Please check ./util/extract_feature_v1.py and ./util/extract_feature_v2.py.

CLOSED 22 Jan 2019: We are fine-tuning our released IR-50 model on our private Asia face data, which will be released soon to facilitate high-performance Asia face recognition.

CLOSED 21 Jan 2019: We are training a better-performing IR-50 model on MS-Celeb-1M_Align_112x112, and will replace the current model soon.


Contents


face.evoLVe for High-Performance Face Recognition

Introduction

💁

  • This repo provides a comprehensive face recognition library for face related analytics & applications, including face alignment (detection, landmark localization, affine transformation, etc.), data processing (e.g., augmentation, data balancing, normalization, etc.), various backbones (e.g., ResNet, IR, IR-SE, ResNeXt, SE-ResNeXt, DenseNet, LightCNN, MobileNet, ShuffleNet, DPN, etc.), various losses (e.g., Softmax, Focal, Center, SphereFace, CosFace, AmSoftmax, ArcFace, Triplet, etc.) and bags of tricks for improving performance (e.g., training refinements, model tweaks, knowledge distillation, etc.).
  • The current distributed training schema with multi-GPUs under PyTorch and other mainstream platforms parallels the backbone across multi-GPUs while relying on a single master to compute the final bottleneck (fully-connected/softmax) layer. This is not an issue for conventional face recognition with moderate number of identities. However, it struggles with large-scale face recognition, which requires recognizing millions of identities in the real world. The master can hardly hold the oversized final layer while the slaves still have redundant computation resource, leading to small-batch training or even failed training. To address this problem, this repo provides a highly-elegant, effective and efficient distributed training schema with multi-GPUs under PyTorch, supporting not only the backbone, but also the head with the fully-connected (softmax) layer, to facilitate high-performance large-scale face recognition.
  • All data before & after alignment, source codes and trained models are provided.
  • This repo can help researchers/engineers develop high-performance deep face recognition models and algorithms quickly for practical use and deployment.

Pre-Requisites

🍰

  • Linux or macOS
  • Python 3.7 (for training & validation) and Python 2.7 (for visualization w/ tensorboardX)
  • PyTorch 1.0 (for traininig & validation, install w/ pip install torch torchvision)
  • MXNet 1.3.1 (optional, for data processing, install w/ pip install mxnet-cu90)
  • TensorFlow 1.12 (optional, for visualization, install w/ pip install tensorflow-gpu)
  • tensorboardX 1.6 (optional, for visualization, install w/ pip install tensorboardX)
  • OpenCV 3.4.5 (install w/ pip install opencv-python)
  • bcolz 1.2.0 (install w/ pip install bcolz)

While not required, for optimal performance it is highly recommended to run the code using a CUDA enabled GPU. We used 4-8 NVIDIA Tesla P40 in parallel.


Usage

📙

  • Clone the repo: git clone https://github.com/ZhaoJ9014/face.evoLVe.PyTorch.git.
  • mkdir data checkpoint log at appropriate directory to store your train/val/test data, checkpoints and training logs.
  • Prepare your train/val/test data (refer to Sec. Data Zoo for publicly available face related databases), and ensure each database folder has the following structure:
    ./data/db_name/
            -> id1/
                -> 1.jpg
                -> ...
            -> id2/
                -> 1.jpg
                -> ...
            -> ...
                -> ...
                -> ...
    
  • Refer to the codes of corresponding sections for specific purposes.

Face Alignment

📐

  • This section is based on the work of MTCNN.
  • Folder: ./align
  • Face detection, landmark localization APIs and visualization toy example with ipython notebook:
    from PIL import Image
    from detector import detect_faces
    from visualization_utils import show_results
    
    img = Image.open('some_img.jpg') # modify the image path to yours
    bounding_boxes, landmarks = detect_faces(img) # detect bboxes and landmarks for all faces in the image
    show_results(img, bounding_boxes, landmarks) # visualize the results
  • Face alignment API (perform face detection, landmark localization and alignment with affine transformations on a whole database folder source_root with the directory structure as demonstrated in Sec. Usage, and store the aligned results to a new folder dest_root with the same directory structure):
    python face_align.py -source_root [source_root] -dest_root [dest_root] -crop_size [crop_size]
    
    # python face_align.py -source_root './data/test' -dest_root './data/test_Aligned' -crop_size 112
    
  • For macOS users, there is no need to worry about *.DS_Store files which may ruin your data, since they will be automatically removed when you run the scripts.
  • Keynotes for customed use: 1) specify the arguments of source_root, dest_root and crop_size to your own values when you run face_align.py; 2) pass your customed min_face_size, thresholds and nms_thresholds values to the detect_faces function of detector.py to match your practical requirements; 3) if you find the speed using face alignment API is a bit slow, you can call face resize API to firstly resize the image whose smaller size is larger than a threshold (specify the arguments of source_root, dest_root and min_side to your own values) before calling the face alignment API:
    python face_resize.py
    

Data Processing

📊

  • Folder: ./balance
  • Remove low-shot data API (remove the low-shot classes with less than min_num samples in the training set root with the directory structure as demonstrated in Sec. Usage for data balance and effective model training):
    python remove_lowshot.py -root [root] -min_num [min_num]
    
    # python remove_lowshot.py -root './data/train' -min_num 10
    
  • Keynotes for customed use: specify the arguments of root and min_num to your own values when you run remove_lowshot.py.
  • We prefer to include other data processing tricks, e.g., augmentation (flip horizontally, scale hue/satuation/brightness with coefficients uniformly drawn from [0.6,1.4], add PCA noise with a coefficient sampled from a normal distribution N(0,0.1), etc.), weighted random sampling, normalization, etc. to the main training script in Sec. Training and Validation to be self-contained.

Training and Validation

  • Folder: ./

  • Configuration API (configurate your overall settings for training & validation) config.py:

    import torch
    
    configurations = {
        1: dict(
            SEED = 1337, # random seed for reproduce results
    
            DATA_ROOT = '/media/pc/6T/jasonjzhao/data/faces_emore', # the parent root where your train/val/test data are stored
            MODEL_ROOT = '/media/pc/6T/jasonjzhao/buffer/model', # the root to buffer your checkpoints
            LOG_ROOT = '/media/pc/6T/jasonjzhao/buffer/log', # the root to log your train/val status
            BACKBONE_RESUME_ROOT = './', # the root to resume training from a saved checkpoint
            HEAD_RESUME_ROOT = './', # the root to resume training from a saved checkpoint
    
            BACKBONE_NAME = 'IR_SE_50', # support: ['ResNet_50', 'ResNet_101', 'ResNet_152', 'IR_50', 'IR_101', 'IR_152', 'IR_SE_50', 'IR_SE_101', 'IR_SE_152']
            HEAD_NAME = 'ArcFace', # support:  ['Softmax', 'ArcFace', 'CosFace', 'SphereFace', 'Am_softmax']
            LOSS_NAME = 'Focal', # support: ['Focal', 'Softmax']
    
            INPUT_SIZE = [112, 112], # support: [112, 112] and [224, 224]
            RGB_MEAN = [0.5, 0.5, 0.5], # for normalize inputs to [-1, 1]
            RGB_STD = [0.5, 0.5, 0.5],
            EMBEDDING_SIZE = 512, # feature dimension
            BATCH_SIZE = 512,
            DROP_LAST = True, # whether drop the last batch to ensure consistent batch_norm statistics
            LR = 0.1, # initial LR
            NUM_EPOCH = 125, # total epoch number (use the firt 1/25 epochs to warm up)
            WEIGHT_DECAY = 5e-4, # do not apply to batch_norm parameters
            MOMENTUM = 0.9,
            STAGES = [35, 65, 95], # epoch stages to decay learning rate
    
            DEVICE = torch.device("cuda:0" if torch.cuda.is_available() else "cpu"),
            MULTI_GPU = True, # flag to use multiple GPUs; if you choose to train with single GPU, you should first run "export CUDA_VISILE_DEVICES=device_id" to specify the GPU card you want to use
            GPU_ID = [0, 1, 2, 3], # specify your GPU ids
            PIN_MEMORY = True,
            NUM_WORKERS = 0,
    ),
    }
  • Train & validation API (all folks about training & validation, i.e., import package, hyperparameters & data loaders, model & loss & optimizer, train & validation & save checkpoint) train.py. Since MS-Celeb-1M serves as an ImageNet in the filed of face recognition, we pre-train the face.evoLVe models on MS-Celeb-1M and perform validation on LFW, CFP_FF, CFP_FP, AgeDB, CALFW, CPLFW and Vggface2_FP. Let's dive into details together step by step.

    • Import necessary packages:
      import torch
      import torch.nn as nn
      import torch.optim as optim
      import torchvision.transforms as transforms
      import torchvision.datasets as datasets
      
      from config import configurations
      from backbone.model_resnet import ResNet_50, ResNet_101, ResNet_152
      from backbone.model_irse import IR_50, IR_101, IR_152, IR_SE_50, IR_SE_101, IR_SE_152
      from head.metrics import ArcFace, CosFace, SphereFace, Am_softmax
      from loss.focal import FocalLoss
      from util.utils import make_weights_for_balanced_classes, get_val_data, separate_irse_bn_paras, separate_resnet_bn_paras, warm_up_lr, schedule_lr, perform_val, get_time, buffer_val, AverageMeter, accuracy
      
      from tensorboardX import SummaryWriter
      from tqdm import tqdm
      import os
    • Initialize hyperparameters:
      cfg = configurations[1]
      
      SEED = cfg['SEED'] # random seed for reproduce results
      torch.manual_seed(SEED)
      
      DATA_ROOT = cfg['DATA_ROOT'] # the parent root where your train/val/test data are stored
      MODEL_ROOT = cfg['MODEL_ROOT'] # the root to buffer your checkpoints
      LOG_ROOT = cfg['LOG_ROOT'] # the root to log your train/val status
      BACKBONE_RESUME_ROOT = cfg['BACKBONE_RESUME_ROOT'] # the root to resume training from a saved checkpoint
      HEAD_RESUME_ROOT = cfg['HEAD_RESUME_ROOT']  # the root to resume training from a saved checkpoint
      
      BACKBONE_NAME = cfg['BACKBONE_NAME'] # support: ['ResNet_50', 'ResNet_101', 'ResNet_152', 'IR_50', 'IR_101', 'IR_152', 'IR_SE_50', 'IR_SE_101', 'IR_SE_152']
      HEAD_NAME = cfg['HEAD_NAME'] # support:  ['Softmax', 'ArcFace', 'CosFace', 'SphereFace', 'Am_softmax']
      LOSS_NAME = cfg['LOSS_NAME'] # support: ['Focal', 'Softmax']
      
      INPUT_SIZE = cfg['INPUT_SIZE']
      RGB_MEAN = cfg['RGB_MEAN'] # for normalize inputs
      RGB_STD = cfg['RGB_STD']
      EMBEDDING_SIZE = cfg['EMBEDDING_SIZE'] # feature dimension
      BATCH_SIZE = cfg['BATCH_SIZE']
      DROP_LAST = cfg['DROP_LAST'] # whether drop the last batch to ensure consistent batch_norm statistics
      LR = cfg['LR'] # initial LR
      NUM_EPOCH = cfg['NUM_EPOCH']
      WEIGHT_DECAY = cfg['WEIGHT_DECAY']
      MOMENTUM = cfg['MOMENTUM']
      STAGES = cfg['STAGES'] # epoch stages to decay learning rate
      
      DEVICE = cfg['DEVICE']
      MULTI_GPU = cfg['MULTI_GPU'] # flag to use multiple GPUs
      GPU_ID = cfg['GPU_ID'] # specify your GPU ids
      PIN_MEMORY = cfg['PIN_MEMORY']
      NUM_WORKERS = cfg['NUM_WORKERS']
      print("=" * 60)
      print("Overall Configurations:")
      print(cfg)
      print("=" * 60)
      
      writer = SummaryWriter(LOG_ROOT) # writer for buffering intermedium results
    • Train & validation data loaders:
      train_transform = transforms.Compose([ # refer to https://pytorch.org/docs/stable/torchvision/transforms.html for more build-in online data augmentation
          transforms.Resize([int(128 * INPUT_SIZE[0] / 112), int(128 * INPUT_SIZE[0] / 112)]), # smaller side resized
          transforms.RandomCrop([INPUT_SIZE[0], INPUT_SIZE[1]]),
          transforms.RandomHorizontalFlip(),
          transforms.ToTensor(),
          transforms.Normalize(mean = RGB_MEAN,
                               std = RGB_STD),
      ])
      
      dataset_train = datasets.ImageFolder(os.path.join(DATA_ROOT, 'imgs'), train_transform)
      
      # create a weighted random sampler to process imbalanced data
      weights = make_weights_for_balanced_classes(dataset_train.imgs, len(dataset_train.classes))
      weights = torch.DoubleTensor(weights)
      sampler = torch.utils.data.sampler.WeightedRandomSampler(weights, len(weights))
      
      train_loader = torch.utils.data.DataLoader(
          dataset_train, batch_size = BATCH_SIZE, sampler = sampler, pin_memory = PIN_MEMORY,
          num_workers = NUM_WORKERS, drop_last = DROP_LAST
      )
      
      NUM_CLASS = len(train_loader.dataset.classes)
      print("Number of Training Classes: {}".format(NUM_CLASS))
      
      lfw, cfp_ff, cfp_fp, agedb, calfw, cplfw, vgg2_fp, lfw_issame, cfp_ff_issame, cfp_fp_issame, agedb_issame, calfw_issame, cplfw_issame, vgg2_fp_issame = get_val_data(DATA_ROOT)
    • Define and initialize model (backbone & head):
      BACKBONE_DICT = {'ResNet_50': ResNet_50(INPUT_SIZE), 
                       'ResNet_101': ResNet_101(INPUT_SIZE), 
                       'ResNet_152': ResNet_152(INPUT_SIZE),
                       'IR_50': IR_50(INPUT_SIZE), 
                       'IR_101': IR_101(INPUT_SIZE), 
                       'IR_152': IR_152(INPUT_SIZE),
                       'IR_SE_50': IR_SE_50(INPUT_SIZE), 
                       'IR_SE_101': IR_SE_101(INPUT_SIZE), 
                       'IR_SE_152': IR_SE_152(INPUT_SIZE)}
      BACKBONE = BACKBONE_DICT[BACKBONE_NAME]
      print("=" * 60)
      print(BACKBONE)
      print("{} Backbone Generated".format(BACKBONE_NAME))
      print("=" * 60)
      
      HEAD_DICT = {'ArcFace': ArcFace(in_features = EMBEDDING_SIZE, out_features = NUM_CLASS, device_id = GPU_ID),
                   'CosFace': CosFace(in_features = EMBEDDING_SIZE, out_features = NUM_CLASS, device_id = GPU_ID),
                   'SphereFace': SphereFace(in_features = EMBEDDING_SIZE, out_features = NUM_CLASS, device_id = GPU_ID),
                   'Am_softmax': Am_softmax(in_features = EMBEDDING_SIZE, out_features = NUM_CLASS, device_id = GPU_ID)}
      HEAD = HEAD_DICT[HEAD_NAME]
      print("=" * 60)
      print(HEAD)
      print("{} Head Generated".format(HEAD_NAME))
      print("=" * 60)
    • Define and initialize loss function:
      LOSS_DICT = {'Focal': FocalLoss(), 
                   'Softmax': nn.CrossEntropyLoss()}
      LOSS = LOSS_DICT[LOSS_NAME]
      print("=" * 60)
      print(LOSS)
      print("{} Loss Generated".format(LOSS_NAME))
      print("=" * 60)
    • Define and initialize optimizer:
      if BACKBONE_NAME.find("IR") >= 0:
          backbone_paras_only_bn, backbone_paras_wo_bn = separate_irse_bn_paras(BACKBONE) # separate batch_norm parameters from others; do not do weight decay for batch_norm parameters to improve the generalizability
          _, head_paras_wo_bn = separate_irse_bn_paras(HEAD)
      else:
          backbone_paras_only_bn, backbone_paras_wo_bn = separate_resnet_bn_paras(BACKBONE) # separate batch_norm parameters from others; do not do weight decay for batch_norm parameters to improve the generalizability
          _, head_paras_wo_bn = separate_resnet_bn_paras(HEAD)
      OPTIMIZER = optim.SGD([{'params': backbone_paras_wo_bn + head_paras_wo_bn, 'weight_decay': WEIGHT_DECAY}, {'params': backbone_paras_only_bn}], lr = LR, momentum = MOMENTUM)
      print("=" * 60)
      print(OPTIMIZER)
      print("Optimizer Generated")
      print("=" * 60)
    • Whether resume from a checkpoint or not:
      if BACKBONE_RESUME_ROOT and HEAD_RESUME_ROOT:
          print("=" * 60)
          if os.path.isfile(BACKBONE_RESUME_ROOT) and os.path.isfile(HEAD_RESUME_ROOT):
              print("Loading Backbone Checkpoint '{}'".format(BACKBONE_RESUME_ROOT))
              BACKBONE.load_state_dict(torch.load(BACKBONE_RESUME_ROOT))
              print("Loading Head Checkpoint '{}'".format(HEAD_RESUME_ROOT))
              HEAD.load_state_dict(torch.load(HEAD_RESUME_ROOT))
          else:
              print("No Checkpoint Found at '{}' and '{}'. Please Have a Check or Continue to Train from Scratch".format(BACKBONE_RESUME_ROOT, HEAD_RESUME_ROOT))
          print("=" * 60)
    • Whether use multi-GPU or not:
      if MULTI_GPU:
          # multi-GPU setting
          BACKBONE = nn.DataParallel(BACKBONE, device_ids = GPU_ID)
          BACKBONE = BACKBONE.to(DEVICE)
      else:
          # single-GPU setting
          BACKBONE = BACKBONE.to(DEVICE)
    • Minor settings prior to training:
      DISP_FREQ = len(train_loader) // 100 # frequency to display training loss & acc
      
      NUM_EPOCH_WARM_UP = NUM_EPOCH // 25  # use the first 1/25 epochs to warm up
      NUM_BATCH_WARM_UP = len(train_loader) * NUM_EPOCH_WARM_UP  # use the first 1/25 epochs to warm up
      batch = 0  # batch index
    • Training & validation & save checkpoint (use the first 1/25 epochs to warm up -- gradually increase LR to the initial value to ensure stable convergence):
      for epoch in range(NUM_EPOCH): # start training process
          
          if epoch == STAGES[0]: # adjust LR for each training stage after warm up, you can also choose to adjust LR manually (with slight modification) once plaueau observed
              schedule_lr(OPTIMIZER)
          if epoch == STAGES[1]:
              schedule_lr(OPTIMIZER)
          if epoch == STAGES[2]:
              schedule_lr(OPTIMIZER)
      
          BACKBONE.train()  # set to training mode
          HEAD.train()
      
          losses = AverageMeter()
          top1 = AverageMeter()
          top5 = AverageMeter()
      
          for inputs, labels in tqdm(iter(train_loader)):
      
              if (epoch + 1 <= NUM_EPOCH_WARM_UP) and (batch + 1 <= NUM_BATCH_WARM_UP): # adjust LR for each training batch during warm up
                  warm_up_lr(batch + 1, NUM_BATCH_WARM_UP, LR, OPTIMIZER)
      
              # compute output
              inputs = inputs.to(DEVICE)
              labels = labels.to(DEVICE).long()
              features = BACKBONE(inputs)
              outputs = HEAD(features, labels)
              loss = LOSS(outputs, labels)
      
              # measure accuracy and record loss
              prec1, prec5 = accuracy(outputs.data, labels, topk = (1, 5))
              losses.update(loss.data.item(), inputs.size(0))
              top1.update(prec1.data.item(), inputs.size(0))
              top5.update(prec5.data.item(), inputs.size(0))
      
              # compute gradient and do SGD step
              OPTIMIZER.zero_grad()
              loss.backward()
              OPTIMIZER.step()
              
              # dispaly training loss & acc every DISP_FREQ
              if ((batch + 1) % DISP_FREQ == 0) and batch != 0:
                  print("=" * 60)
                  print('Epoch {}/{} Batch {}/{}\t'
                        'Training Loss {loss.val:.4f} ({loss.avg:.4f})\t'
                        'Training Prec@1 {top1.val:.3f} ({top1.avg:.3f})\t'
                        'Training Prec@5 {top5.val:.3f} ({top5.avg:.3f})'.format(
                      epoch + 1, NUM_EPOCH, batch + 1, len(train_loader) * NUM_EPOCH, loss = losses, top1 = top1, top5 = top5))
                  print("=" * 60)
      
              batch += 1 # batch index
      
          # training statistics per epoch (buffer for visualization)
          epoch_loss = losses.avg
          epoch_acc = top1.avg
          writer.add_scalar("Training_Loss", epoch_loss, epoch + 1)
          writer.add_scalar("Training_Accuracy", epoch_acc, epoch + 1)
          print("=" * 60)
          print('Epoch: {}/{}\t'
                'Training Loss {loss.val:.4f} ({loss.avg:.4f})\t'
                'Training Prec@1 {top1.val:.3f} ({top1.avg:.3f})\t'
                'Training Prec@5 {top5.val:.3f} ({top5.avg:.3f})'.format(
              epoch + 1, NUM_EPOCH, loss = losses, top1 = top1, top5 = top5))
          print("=" * 60)
      
          # perform validation & save checkpoints per epoch
          # validation statistics per epoch (buffer for visualization)
          print("=" * 60)
          print("Perform Evaluation on LFW, CFP_FF, CFP_FP, AgeDB, CALFW, CPLFW and VGG2_FP, and Save Checkpoints...")
          accuracy_lfw, best_threshold_lfw, roc_curve_lfw = perform_val(MULTI_GPU, DEVICE, EMBEDDING_SIZE, BATCH_SIZE, BACKBONE, lfw, lfw_issame)
          buffer_val(writer, "LFW", accuracy_lfw, best_threshold_lfw, roc_curve_lfw, epoch + 1)
          accuracy_cfp_ff, best_threshold_cfp_ff, roc_curve_cfp_ff = perform_val(MULTI_GPU, DEVICE, EMBEDDING_SIZE, BATCH_SIZE, BACKBONE, cfp_ff, cfp_ff_issame)
          buffer_val(writer, "CFP_FF", accuracy_cfp_ff, best_threshold_cfp_ff, roc_curve_cfp_ff, epoch + 1)
          accuracy_cfp_fp, best_threshold_cfp_fp, roc_curve_cfp_fp = perform_val(MULTI_GPU, DEVICE, EMBEDDING_SIZE, BATCH_SIZE, BACKBONE, cfp_fp, cfp_fp_issame)
          buffer_val(writer, "CFP_FP", accuracy_cfp_fp, best_threshold_cfp_fp, roc_curve_cfp_fp, epoch + 1)
          accuracy_agedb, best_threshold_agedb, roc_curve_agedb = perform_val(MULTI_GPU, DEVICE, EMBEDDING_SIZE, BATCH_SIZE, BACKBONE, agedb, agedb_issame)
          buffer_val(writer, "AgeDB", accuracy_agedb, best_threshold_agedb, roc_curve_agedb, epoch + 1)
          accuracy_calfw, best_threshold_calfw, roc_curve_calfw = perform_val(MULTI_GPU, DEVICE, EMBEDDING_SIZE, BATCH_SIZE, BACKBONE, calfw, calfw_issame)
          buffer_val(writer, "CALFW", accuracy_calfw, best_threshold_calfw, roc_curve_calfw, epoch + 1)
          accuracy_cplfw, best_threshold_cplfw, roc_curve_cplfw = perform_val(MULTI_GPU, DEVICE, EMBEDDING_SIZE, BATCH_SIZE, BACKBONE, cplfw, cplfw_issame)
          buffer_val(writer, "CPLFW", accuracy_cplfw, best_threshold_cplfw, roc_curve_cplfw, epoch + 1)
          accuracy_vgg2_fp, best_threshold_vgg2_fp, roc_curve_vgg2_fp = perform_val(MULTI_GPU, DEVICE, EMBEDDING_SIZE, BATCH_SIZE, BACKBONE, vgg2_fp, vgg2_fp_issame)
          buffer_val(writer, "VGGFace2_FP", accuracy_vgg2_fp, best_threshold_vgg2_fp, roc_curve_vgg2_fp, epoch + 1)
          print("Epoch {}/{}, Evaluation: LFW Acc: {}, CFP_FF Acc: {}, CFP_FP Acc: {}, AgeDB Acc: {}, CALFW Acc: {}, CPLFW Acc: {}, VGG2_FP Acc: {}".format(epoch + 1, NUM_EPOCH, accuracy_lfw, accuracy_cfp_ff, accuracy_cfp_fp, accuracy_agedb, accuracy_calfw, accuracy_cplfw, accuracy_vgg2_fp))
          print("=" * 60)
      
          # save checkpoints per epoch
          if MULTI_GPU:
              torch.save(BACKBONE.module.state_dict(), os.path.join(MODEL_ROOT, "Backbone_{}_Epoch_{}_Batch_{}_Time_{}_checkpoint.pth".format(BACKBONE_NAME, epoch + 1, batch, get_time())))
              torch.save(HEAD.state_dict(), os.path.join(MODEL_ROOT, "Head_{}_Epoch_{}_Batch_{}_Time_{}_checkpoint.pth".format(HEAD_NAME, epoch + 1, batch, get_time())))
          else:
              torch.save(BACKBONE.state_dict(), os.path.join(MODEL_ROOT, "Backbone_{}_Epoch_{}_Batch_{}_Time_{}_checkpoint.pth".format(BACKBONE_NAME, epoch + 1, batch, get_time())))
              torch.save(HEAD.state_dict(), os.path.join(MODEL_ROOT, "Head_{}_Epoch_{}_Batch_{}_Time_{}_checkpoint.pth".format(HEAD_NAME, epoch + 1, batch, get_time())))
  • Now, you can start to play with face.evoLVe and run train.py. User friendly information will popped out on your terminal:

    • About overall configuration:

    • About number of training classes:

    • About backbone details:

    • About head details:

    • About loss details:

    • About optimizer details:

    • About resume training:

    • About training status & statistics (when batch index reachs DISP_FREQ or at the end of each epoch):

    • About validation statistics & save checkpoints (at the end of each epoch):

  • Monitor on-the-fly GPU occupancy with watch -d -n 0.01 nvidia-smi.

  • Please refer to Sec. Model Zoo for specific model weights and corresponding performance.

  • Feature extraction API (extract features from pre-trained models) ./util/extract_feature_v1.py (implemented with PyTorch build-in functions) and ./util/extract_feature_v2.py (implemented with OpenCV).

  • Visualize training & validation statistics with tensorboardX (see Sec. Model Zoo):

    tensorboard --logdir /media/pc/6T/jasonjzhao/buffer/log
    

Data Zoo

🐯

Database Version #Identity #Image #Frame #Video Download Link
LFW Raw 5,749 13,233 - - Google Drive, Baidu Drive
LFW Align_250x250 5,749 13,233 - - Google Drive, Baidu Drive
LFW Align_112x112 5,749 13,233 - - Google Drive, Baidu Drive
CALFW Raw 4,025 12,174 - - Google Drive, Baidu Drive
CALFW Align_112x112 4,025 12,174 - - Google Drive, Baidu Drive
CPLFW Raw 3,884 11,652 - - Google Drive, Baidu Drive
CPLFW Align_112x112 3,884 11,652 - - Google Drive, Baidu Drive
CASIA-WebFace Raw_v1 10,575 494,414 - - Baidu Drive
CASIA-WebFace Raw_v2 10,575 494,414 - - Google Drive, Baidu Drive
CASIA-WebFace Clean 10,575 455,594 - - Google Drive, Baidu Drive
MS-Celeb-1M Clean 100,000 5,084,127 - - Google Drive
MS-Celeb-1M Align_112x112 85,742 5,822,653 - - Google Drive
Vggface2 Clean 8,631 3,086,894 - - Google Drive
Vggface2_FP Align_112x112 - - - - Google Drive, Baidu Drive
AgeDB Raw 570 16,488 - - Google Drive, Baidu Drive
AgeDB Align_112x112 570 16,488 - - Google Drive, Baidu Drive
IJB-A Clean 500 5,396 20,369 2,085 Google Drive, Baidu Drive
IJB-B Raw 1,845 21,798 55,026 7,011 Google Drive
CFP Raw 500 7,000 - - Google Drive, Baidu Drive
CFP Align_112x112 500 7,000 - - Google Drive, Baidu Drive
Umdfaces Align_112x112 8,277 367,888 - - Google Drive, Baidu Drive
CelebA Raw 10,177 202,599 - - Google Drive, Baidu Drive
CACD-VS Raw 2,000 163,446 - - Google Drive, Baidu Drive
YTF Align_344x344 1,595 - 3,425 621,127 Google Drive, Baidu Drive
DeepGlint Align_112x112 180,855 6,753,545 - - Google Drive
UTKFace Align_200x200 - 23,708 - - Google Drive, Baidu Drive
BUAA-VisNir Align_287x287 150 5,952 - - Baidu Drive, PW: xmbc
CASIA NIR-VIS 2.0 Align_128x128 725 17,580 - - Baidu Drive, PW: 883b
Oulu-CASIA Raw 80 65,000 - - Baidu Drive, PW: xxp5
NUAA-ImposterDB Raw 15 12,614 - - Baidu Drive, PW: if3n
CASIA-SURF Raw 1,000 - - 21,000 Baidu Drive, PW: izb3
CASIA-FASD Raw 50 - - 600 Baidu Drive, PW: h5un
CASIA-MFSD Raw 50 - - 600
Replay-Attack Raw 50 - - 1,200
WebFace260M Raw 24M 2M - https://www.face-benchmark.org/
  • Remark: unzip CASIA-WebFace clean version with
    unzip casia-maxpy-clean.zip    
    cd casia-maxpy-clean    
    zip -F CASIA-maxpy-clean.zip --out CASIA-maxpy-clean_fix.zip    
    unzip CASIA-maxpy-clean_fix.zip
    
  • Remark: after unzip, get image data & pair ground truths from AgeDB, CFP, LFW and VGGFace2_FP align_112x112 versions with
    import numpy as np
    import bcolz
    import os
    
    def get_pair(root, name):
        carray = bcolz.carray(rootdir = os.path.join(root, name), mode='r')
        issame = np.load('{}/{}_list.npy'.format(root, name))
        return carray, issame
    
    def get_data(data_root):
        agedb_30, agedb_30_issame = get_pair(data_root, 'agedb_30')
        cfp_fp, cfp_fp_issame = get_pair(data_root, 'cfp_fp')
        lfw, lfw_issame = get_pair(data_root, 'lfw')
        vgg2_fp, vgg2_fp_issame = get_pair(data_root, 'vgg2_fp')
        return agedb_30, cfp_fp, lfw, vgg2_fp, agedb_30_issame, cfp_fp_issame, lfw_issame, vgg2_fp_issame
    
    agedb_30, cfp_fp, lfw, vgg2_fp, agedb_30_issame, cfp_fp_issame, lfw_issame, vgg2_fp_issame = get_data(DATA_ROOT)
  • Remark: We share MS-Celeb-1M_Top1M_MID2Name.tsv (Google Drive, Baidu Drive), VGGface2_ID2Name.csv (Google Drive, Baidu Drive), VGGface2_FaceScrub_Overlap.txt (Google Drive, Baidu Drive), VGGface2_LFW_Overlap.txt (Google Drive, Baidu Drive), CASIA-WebFace_ID2Name.txt (Google Drive, Baidu Drive), CASIA-WebFace_FaceScrub_Overlap.txt (Google Drive, Baidu Drive), CASIA-WebFace_LFW_Overlap.txt (Google Drive, Baidu Drive), FaceScrub_Name.txt (Google Drive, Baidu Drive), LFW_Name.txt (Google Drive, Baidu Drive), LFW_Log.txt (Google Drive, Baidu Drive) to help researchers/engineers quickly remove the overlapping parts between their own private datasets and the public datasets.
  • Due to release license issue, for other face related databases, please make contact with us in person for more details.

Model Zoo

🐒

  • Model

    Backbone Head Loss Training Data Download Link
    IR-50 ArcFace Focal MS-Celeb-1M_Align_112x112 Google Drive, Baidu Drive
    • Setting

      INPUT_SIZE: [112, 112]; RGB_MEAN: [0.5, 0.5, 0.5]; RGB_STD: [0.5, 0.5, 0.5]; BATCH_SIZE: 512 (drop the last batch to ensure consistent batch_norm statistics); Initial LR: 0.1; NUM_EPOCH: 120; WEIGHT_DECAY: 5e-4 (do not apply to batch_norm parameters); MOMENTUM: 0.9; STAGES: [30, 60, 90]; Augmentation: Random Crop + Horizontal Flip; Imbalanced Data Processing: Weighted Random Sampling; Solver: SGD; GPUs: 4 NVIDIA Tesla P40 in Parallel
      
    • Training & validation statistics

    • Performance

      LFW CFP_FF CFP_FP AgeDB CALFW CPLFW Vggface2_FP
      99.78 99.69 98.14 97.53 95.87 92.45 95.22
  • Model

    Backbone Head Loss Training Data Download Link
    IR-50 ArcFace Focal Private Asia Face Data Google Drive, Baidu Drive
    • Setting

      INPUT_SIZE: [112, 112]; RGB_MEAN: [0.5, 0.5, 0.5]; RGB_STD: [0.5, 0.5, 0.5]; BATCH_SIZE: 1024 (drop the last batch to ensure consistent batch_norm statistics); Initial LR: 0.01 (initialize weights from the above model pre-trained on MS-Celeb-1M_Align_112x112); NUM_EPOCH: 80; WEIGHT_DECAY: 5e-4 (do not apply to batch_norm parameters); MOMENTUM: 0.9; STAGES: [20, 40, 60]; Augmentation: Random Crop + Horizontal Flip; Imbalanced Data Processing: Weighted Random Sampling; Solver: SGD; GPUs: 8 NVIDIA Tesla P40 in Parallel
      
    • Performance (please perform evaluation on your own Asia face benchmark dataset)

  • Model

    Backbone Head Loss Training Data Download Link
    IR-152 ArcFace Focal MS-Celeb-1M_Align_112x112 Baidu Drive, PW: b197
    • Setting

      INPUT_SIZE: [112, 112]; RGB_MEAN: [0.5, 0.5, 0.5]; RGB_STD: [0.5, 0.5, 0.5]; BATCH_SIZE: 256 (drop the last batch to ensure consistent batch_norm statistics); Initial LR: 0.01; NUM_EPOCH: 120; WEIGHT_DECAY: 5e-4 (do not apply to batch_norm parameters); MOMENTUM: 0.9; STAGES: [30, 60, 90]; Augmentation: Random Crop + Horizontal Flip; Imbalanced Data Processing: Weighted Random Sampling; Solver: SGD; GPUs: 4 NVIDIA Geforce RTX 2080 Ti in Parallel
      
    • Training & validation statistics

    • Performance

      LFW CFP_FF CFP_FP AgeDB CALFW CPLFW Vggface2_FP
      99.82 99.83 98.37 98.07 96.03 93.05 95.50

Achievement

🎊

  • 2017 No.1 on ICCV 2017 MS-Celeb-1M Large-Scale Face Recognition Hard Set/Random Set/Low-Shot Learning Challenges. WeChat News, NUS ECE News, NUS ECE Poster, Award Certificate for Track-1, Award Certificate for Track-2, Award Ceremony.

  • 2017 No.1 on National Institute of Standards and Technology (NIST) IARPA Janus Benchmark A (IJB-A) Unconstrained Face Verification challenge and Identification challenge. WeChat News.

  • State-of-the-art performance on

    • MS-Celeb-1M (Challenge1 Hard Set Coverage@P=0.95: 79.10%; Challenge1 Random Set Coverage@P=0.95: 87.50%; Challenge2 Development Set Coverage@P=0.99: 100.00%; Challenge2 Base Set Top 1 Accuracy: 99.74%; Challenge2 Novel Set Coverage@P=0.99: 99.01%).
    • IJB-A (1:1 Veification TAR@FAR=0.1: 99.6%±0.1%; 1:1 Veification TAR@FAR=0.01: 99.1%±0.2%; 1:1 Veification TAR@FAR=0.001: 97.9%±0.4%; 1:N Identification FNIR@FPIR=0.1: 1.3%±0.3%; 1:N Identification FNIR@FPIR=0.01: 5.4%±4.7%; 1:N Identification Rank1 Accuracy: 99.2%±0.1%; 1:N Identification Rank5 Accuracy: 99.7%±0.1%; 1:N Identification Rank10 Accuracy: 99.8%±0.1%).
    • IJB-C (1:1 Veification TAR@FAR=1e-5: 82.6%).
    • Labeled Faces in the Wild (LFW) (Accuracy: 99.85%±0.217%).
    • Celebrities in Frontal-Profile (CFP) (Frontal-Profile Accuracy: 96.01%±0.84%; Frontal-Profile EER: 4.43%±1.04%; Frontal-Profile AUC: 99.00%±0.35%; Frontal-Frontal Accuracy: 99.64%±0.25%; Frontal-Frontal EER: 0.54%±0.37%; Frontal-Frontal AUC: 99.98%±0.03%).
    • CMU Multi-PIE (Rank1 Accuracy Setting-1 under ±90°: 76.12%; Rank1 Accuracy Setting-2 under ±90°: 86.73%).
    • MORPH Album2 (Rank1 Accuracy Setting-1: 99.65%; Rank1 Accuracy Setting-2: 99.26%).
    • CACD-VS (Accuracy: 99.76%).
    • FG-NET (Rank1 Accuracy: 93.20%).

Acknowledgement

👬


Donation

💰

  • Your donation is highly welcomed to help us further develop face.evoLVe to better facilitate more cutting-edge researches and applications on facial analytics and human-centric multi-media understanding. The donation QR code via Wechat is as below. Appreciate it very much ❤️


Citation

📑

  • Please consult and consider citing the following papers:

    @article{wang2021face,
    title={Face. evoLVe: A High-Performance Face Recognition Library},
    author={Wang, Qingzhong and Zhang, Pengfei and Xiong, Haoyi and Zhao, Jian},
    journal={arXiv preprint arXiv:2107.08621},
    year={2021}
    }
    
    
    @article{tu2021image,
    title={Image-to-Video Generation via 3D Facial Dynamics},
    author={Tu, Xiaoguang and Zou, Yingtian and Zhao, Jian and Ai, Wenjie and Dong, Jian and Yao, Yuan and Wang, Zhikang and Guo, Guodong and Li, Zhifeng and Liu, Wei and others},
    journal={T-CSVT},
    year={2021}
    }
    
    
    @article{tu2021joint,
    title={Joint Face Image Restoration and Frontalization for Recognition},
    author={Tu, Xiaoguang and Zhao, Jian and Liu, Qiankun and Ai, Wenjie and Guo, Guodong and Li, Zhifeng and Liu, Wei and Feng, Jiashi},
    journal={T-CSVT},
    year={2021}
    }
    
    
    @article{zhao2020towards,
    title={Towards age-invariant face recognition},
    author={Zhao, Jian and Yan, Shuicheng and Feng, Jiashi},
    journal={T-PAMI},
    year={2020}
    }
    
    
    @article{liang2020fine,
    title={Fine-grained facial expression recognition in the wild},
    author={Liang, Liqian and Lang, Congyan and Li, Yidong and Feng, Songhe and Zhao, Jian},
    journal={T-IFS},
    pages={482--494},
    year={2020}
    }
    
    
    @article{tu2020learning,
    title={Learning generalizable and identity-discriminative representations for face anti-spoofing},
    author={Tu, Xiaoguang and Ma, Zheng and Zhao, Jian and Du, Guodong and Xie, Mei and Feng, Jiashi},
    journal={T-IST},
    pages={1--19},
    year={2020}
    }
    
    
    @article{tu20203d,
    title={3D face reconstruction from a single image assisted by 2D face images in the wild},
    author={Tu, Xiaoguang and Zhao, Jian and Xie, Mei and Jiang, Zihang and Balamurugan, Akshaya and Luo, Yao and Zhao, Yang and He, Lingxiao and Ma, Zheng and Feng, Jiashi},
    journal={T-MM},
    year={2020}
    }
    
    
    @inproceedings{wang2020learning,
    title={Learning to Detect Head Movement in Unconstrained Remote Gaze Estimation in the Wild},
    author={Wang, Zhecan and Zhao, Jian and Lu, Cheng and Yang, Fan and Huang, Han and Guo, Yandong and others},
    booktitle={WACV},
    pages={3443--3452},
    year={2020}
    }
    
    
    @article{zhao2019recognizing,
    title={Recognizing Profile Faces by Imagining Frontal View},
    author={Zhao, Jian and Xing, Junliang and Xiong, Lin and Yan, Shuicheng and Feng, Jiashi},
    journal={IJCV},
    pages={1--19},
    year={2019}
    }
    
    
    @article{kong2019cross,
    title={Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation},
    author={Kong, Hanyang and Zhao, Jian and Tu, Xiaoguang and Xing, Junliang and Shen, Shengmei and Feng, Jiashi},
    journal={arXiv preprint arXiv:1905.10777},
    year={2019}
    }
    
    
    @article{tu2019joint,
    title={Joint 3D face reconstruction and dense face alignment from a single image with 2D-assisted self-supervised learning},
    author={Tu, Xiaoguang and Zhao, Jian and Jiang, Zihang and Luo, Yao and Xie, Mei and Zhao, Yang and He, Linxiao and Ma, Zheng and Feng, Jiashi},
    journal={arXiv preprint arXiv:1903.09359},
    year={2019}
    }     
    
    
    @inproceedings{zhao2019multi,
    title={Multi-Prototype Networks for Unconstrained Set-based Face Recognition},
    author={Zhao, Jian and Li, Jianshu and Tu, Xiaoguang and Zhao, Fang and Xin, Yuan and Xing, Junliang and Liu, Hengzhu and Yan, Shuicheng and Feng, Jiashi},
    booktitle={IJCAI},
    year={2019}
    }
    
    
    @inproceedings{zhao2019look,
    title={Look Across Elapse: Disentangled Representation Learning and Photorealistic Cross-Age Face Synthesis for Age-Invariant Face Recognition},
    author={Zhao, Jian and Cheng, Yu and Cheng, Yi and Yang, Yang and Lan, Haochong and Zhao, Fang and Xiong, Lin and Xu, Yan and Li, Jianshu and Pranata, Sugiri and others},
    booktitle={AAAI},
    year={2019}
    }
    
    
    @article{tu2019joint,
    title={Joint 3D Face Reconstruction and Dense Face Alignment from A Single Image with 2D-Assisted Self-Supervised Learning},
    author={Tu, Xiaoguang and Zhao, Jian and Jiang, Zihang and Luo, Yao and Xie, Mei and Zhao, Yang and He, Linxiao and Ma, Zheng and Feng, Jiashi},
    journal={arXiv preprint arXiv:1903.09359},
    year={2019}
    }
    
    
    @article{tu2019learning,
    title={Learning Generalizable and Identity-Discriminative Representations for Face Anti-Spoofing},
    author={Tu, Xiaoguang and Zhao, Jian and Xie, Mei and Du, Guodong and Zhang, Hengsheng and Li, Jianshu and Ma, Zheng and Feng, Jiashi},
    journal={arXiv preprint arXiv:1901.05602},
    year={2019}
    }
    
    
    @article{zhao20183d,
    title={3D-Aided Dual-Agent GANs for Unconstrained Face Recognition},
    author={Zhao, Jian and Xiong, Lin and Li, Jianshu and Xing, Junliang and Yan, Shuicheng and Feng, Jiashi},
    journal={T-PAMI},
    year={2018}
    }
    
    
    @inproceedings{zhao2018towards,
    title={Towards Pose Invariant Face Recognition in the Wild},
    author={Zhao, Jian and Cheng, Yu and Xu, Yan and Xiong, Lin and Li, Jianshu and Zhao, Fang and Jayashree, Karlekar and Pranata,         Sugiri and Shen, Shengmei and Xing, Junliang and others},
    booktitle={CVPR},
    pages={2207--2216},
    year={2018}
    }
    
    
    @inproceedings{zhao3d,
    title={3D-Aided Deep Pose-Invariant Face Recognition},
    author={Zhao, Jian and Xiong, Lin and Cheng, Yu and Cheng, Yi and Li, Jianshu and Zhou, Li and Xu, Yan and Karlekar, Jayashree and       Pranata, Sugiri and Shen, Shengmei and others},
    booktitle={IJCAI},
    pages={1184--1190},
    year={2018}
    }
    
    
    @inproceedings{zhao2018dynamic,
    title={Dynamic Conditional Networks for Few-Shot Learning},
    author={Zhao, Fang and Zhao, Jian and Yan, Shuicheng and Feng, Jiashi},
    booktitle={ECCV},
    pages={19--35},
    year={2018}
    }
    
    
    @inproceedings{zhao2017dual,
    title={Dual-agent gans for photorealistic and identity preserving profile face synthesis},
    author={Zhao, Jian and Xiong, Lin and Jayashree, Panasonic Karlekar and Li, Jianshu and Zhao, Fang and Wang, Zhecan and Pranata,           Panasonic Sugiri and Shen, Panasonic Shengmei and Yan, Shuicheng and Feng, Jiashi},
    booktitle={NeurIPS},
    pages={66--76},
    year={2017}
    }
    
    
    @inproceedings{zhao122017marginalized,
    title={Marginalized cnn: Learning deep invariant representations},
    author={Zhao12, Jian and Li, Jianshu and Zhao, Fang and Yan13, Shuicheng and Feng, Jiashi},
    booktitle={BMVC},
    year={2017}
    }
    
    
    @inproceedings{cheng2017know,
    title={Know you at one glance: A compact vector representation for low-shot learning},
    author={Cheng, Yu and Zhao, Jian and Wang, Zhecan and Xu, Yan and Jayashree, Karlekar and Shen, Shengmei and Feng, Jiashi},
    booktitle={ICCVW},
    pages={1924--1932},
    year={2017}
    }
    
    
    @inproceedings{xu2017high,
    title={High performance large scale face recognition with multi-cognition softmax and feature retrieval},
    author={Xu, Yan and Cheng, Yu and Zhao, Jian and Wang, Zhecan and Xiong, Lin and Jayashree, Karlekar and Tamura, Hajime and Kagaya, Tomoyuki and Shen, Shengmei and Pranata, Sugiri and others},
    booktitle={ICCVW},
    pages={1898--1906},
    year={2017}
    }
    
    
    @inproceedings{wangconditional,
    title={Conditional Dual-Agent GANs for Photorealistic and Annotation Preserving Image Synthesis},
    author={Wang, Zhecan and Zhao, Jian and Cheng, Yu and Xiao, Shengtao and Li, Jianshu and Zhao, Fang and Feng, Jiashi and Kassim, Ashraf},
    booktitle={BMVCW},
    }
    
    
    @inproceedings{li2017integrated,
    title={Integrated face analytics networks through cross-dataset hybrid training},
    author={Li, Jianshu and Xiao, Shengtao and Zhao, Fang and Zhao, Jian and Li, Jianan and Feng, Jiashi and Yan, Shuicheng and Sim, Terence},
    booktitle={ACM MM},
    pages={1531--1539},
    year={2017}
    }
    
    
    @article{xiong2017good,
    title={A good practice towards top performance of face recognition: Transferred deep feature fusion},
    author={Xiong, Lin and Karlekar, Jayashree and Zhao, Jian and Cheng, Yi and Xu, Yan and Feng, Jiashi and Pranata, Sugiri and Shen, Shengmei},
    journal={arXiv preprint arXiv:1704.00438},
    year={2017}
    }
    
    
    @article{zhao2017robust,
    title={Robust lstm-autoencoders for face de-occlusion in the wild},
    author={Zhao, Fang and Feng, Jiashi and Zhao, Jian and Yang, Wenhan and Yan, Shuicheng},
    journal={T-IP},
    volume={27},
    number={2},
    pages={778--790},
    year={2017}
    }
    
    
    @inproceedings{li2016robust,
    title={Robust face recognition with deep multi-view representation learning},
    author={Li, Jianshu and Zhao, Jian and Zhao, Fang and Liu, Hao and Li, Jing and Shen, Shengmei and Feng, Jiashi and Sim, Terence},
    booktitle={ACM MM},
    pages={1068--1072},
    year={2016}
    }
    
Comments
  • Whats the password for CASIA-SURF(RAW)?

    Whats the password for CASIA-SURF(RAW)?

    Hi, thank you very much for your great contribution. I have downloaded the CASIA-SURF dataset, However, upon extracting it, it requires a password! I tried izb3 which is given for the Baidu drive, but that doesnt work. So I'd really apprecaite it if you could kindly share with us the password for extraction? Thanks a lot

    opened by Coderx7 10
  • CASIA-SURF‘s unzipping password

    CASIA-SURF‘s unzipping password

    Hi, I have downloaded the CASIA-SURF data from the given link. But when I unzip the file, it needs unzipping password. Can you please provide the password? Thanks!

    opened by WH2016 10
  • CASIA-Surf

    CASIA-Surf

    This is not really an issue.

    Can someone please upload CASIA-SURF dataset to other fileshare system? I am not being able to download it, since I cannot create Baidu account.

    Really appreciate it.

    opened by mlourencoeb 8
  • Asking for meta and sizes

    Asking for meta and sizes

    i am trying to train the network on lfw,

    my folder structure looks like

    $RootFolder/data/train, $RootFolder/data/val, $RootFolder/data/test

    but when i run train.py , i get

    FileNotFoundError: [Errno 2] No such file or directory: '/media/ryan/shakira/face.evoLVe.PyTorch/data/lfw/meta/sizes'

    Am i missing something?

    Thanks in advance.

    opened by DecentMakeover 8
  • there are lot of errors in this git lib for example utlits.py there are extra  parenthesis.after solved nearly 50 problems i got stuck here

    there are lot of errors in this git lib for example utlits.py there are extra parenthesis.after solved nearly 50 problems i got stuck here

     python train.py
    ============================================================
    Overall Configurations:
    {'SEED': 1337, 'DATA_ROOT': '/media/mustafa/ubuntu_backup/face/face.evoLVe.PyTorch/alogn_faces', 'MODEL_ROOT': '/media/mustafa/ubuntu_backup/face/face.evoLVe.PyTorch/model', 'LOG_ROOT': '/media/mustafa/ubuntu_backup/face/face.evoLVe.PyTorch/log', 'BACKBONE_RESUME_ROOT': './', 'HEAD_RESUME_ROOT': './', 'BACKBONE_NAME': 'IR_SE_50', 'HEAD_NAME': 'ArcFace', 'LOSS_NAME': 'Focal', 'INPUT_SIZE': [112, 112], 'RGB_MEAN': [0.5, 0.5, 0.5], 'RGB_STD': [0.5, 0.5, 0.5], 'EMBEDDING_SIZE': 512, 'BATCH_SIZE': 512, 'DROP_LAST': True, 'LR': 0.1, 'NUM_EPOCH': 125, 'WEIGHT_DECAY': 0.0005, 'MOMENTUM': 0.9, 'STAGES': [35, 65, 95], 'DEVICE': device(type='cuda', index=0), 'MULTI_GPU': True, 'GPU_ID': [0, 1, 2, 3], 'PIN_MEMORY': True, 'NUM_WORKERS': 0}
    ============================================================
    Traceback (most recent call last):
      File "train.py", line 73, in <module>
        weights = make_weights_for_balanced_classes(dataset_train.imgs, len(dataset_train.classes))
      File "/media/mustafa/ubuntu_backup/face/face.evoLVe.PyTorch/util/utils.py", line 47, in make_weights_for_balanced_classes
        weight_per_class[i] = N / float(count[i])
    ZeroDivisionError: float division by zero
    
    opened by tomriddle54 8
  • Training Loss 19.6988. Can't continue to fall

    Training Loss 19.6988. Can't continue to fall

    Hope you can help me! this is train log.

    configurations = { 1: dict( SEED = 1337, # random seed for reproduce results

        DATA_ROOT = '/lustre/users/chenguang/data/face', # the parent root where your train/val/test data are stored
        MODEL_ROOT = './model/IR_50', # the root to buffer your checkpoints
        LOG_ROOT = './log', # the root to log your train/val status
        BACKBONE_RESUME_ROOT = './weight/AsiaFace_bh-ir50/backbone_ir50_asia.pth', # the root to resume training from a saved checkpoint
        HEAD_RESUME_ROOT = './', # the root to resume training from a saved checkpoint
    
        BACKBONE_NAME = 'IR_50', # support: ['ResNet_50', 'ResNet_101', 'ResNet_152', 'IR_50', 'IR_101', 'IR_152', 'IR_SE_50', 'IR_SE_101', 'IR_SE_152']
        HEAD_NAME = 'ArcFace', # support:  ['Softmax', 'ArcFace', 'CosFace', 'SphereFace', 'Am_softmax']
        LOSS_NAME = 'Focal', # support: ['Focal', 'Softmax']
    
        INPUT_SIZE = [112, 112], # support: [112, 112] and [224, 224]
        RGB_MEAN = [0.5, 0.5, 0.5], # for normalize inputs to [-1, 1]
        RGB_STD = [0.5, 0.5, 0.5],
        EMBEDDING_SIZE = 512, # feature dimension
        BATCH_SIZE = 256,   # 512
        DROP_LAST = True, # whether drop the last batch to ensure consistent batch_norm statistics
        LR = 0.1, # initial LR
        NUM_EPOCH = 125, # total epoch number (use the firt 1/25 epochs to warm up)
        WEIGHT_DECAY = 5e-4, # do not apply to batch_norm parameters
        MOMENTUM = 0.9,
        STAGES = [35, 65, 95], # epoch stages to decay learning rate
    
        DEVICE = torch.device("cuda:0" if torch.cuda.is_available() else "cpu"),
        MULTI_GPU = True, # flag to use multiple GPUs; if you choose to train with single GPU, you should first run "export CUDA_VISILE_DEVICES=device_id" to specify the GPU card you want to use
        GPU_ID = [0, 1, 2, 3], # specify your GPU ids
        PIN_MEMORY = True,
        NUM_WORKERS = 0,
    

    ), }

    **Epoch 59/125 Batch 112176/237750 Training Loss 19.4058 (19.3695) Training Prec@1 0.000 (0.000) Training Prec@5 0.000 (0.000)

    ============================================================ Epoch 59/125 Batch 112195/237750 Training Loss 19.2279 (19.3684) Training Prec@1 0.000 (0.000) Training Prec@5 0.000 (0.000)

    ============================================================ Epoch 59/125 Batch 112214/237750 Training Loss 19.6988 (19.3685) Training Prec@1 0.000 (0.000) Training Prec@5 0.000 (0.000)

    ============================================================ Epoch: 59/125 Training Loss 19.4309 (19.3681) Training Prec@1 0.000 (0.000) Training Prec@5 0.000 (0.000)

    100%|██████████| 1902/1902 [21:11<00:00, 1.50it/s] 32%|███▏ | 616/1902 [06:52<============================================================ Perform Evaluation on LFW, CFP_FF, CFP_FP, AgeDB, CALFW, CPLFW and VGG2_FP, and Save Checkpoints... Epoch 59/125, Evaluation: LFW Acc: 0.974, CPLFW Acc: 0.8041666666666666 **

    opened by chen1234520 5
  • strange loss and presion

    strange loss and presion

    @ZhaoJ9014 I used the default parameters in your config and trained model in the dataset(Asian-Celeb 93,979 ids with 2,830,146 aligned images) QQ截图20191205102511 ,however, the loss was still high and Precision was 0 after epoch 72,what should I do ? is this normal?or is it the matter of parameters in config?

    opened by 3-leaves-grass 5
  • face alignment lfw

    face alignment lfw

    If I perform face alignment using MTCNN on the lfw images, then I end up with fewer than 6000 pairs due to MTCNN being unable to detect/align each one of the lfw images.

    How come when I download your LFW (Align_112x112) from Google drive I see that there are 6000 pairs? How did you get MTCNN to successfully align each of the LFW images?

    Thanks.

    opened by jeroneandrews-sony 5
  • How to disable this user warning?

    How to disable this user warning?

    When run detector to show result on detect result, there is an warning:

    /align/get_nets.py:70: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
      a = F.softmax(a)
    detector.py:82: UserWarning: volatile was removed and now has no effect. Use `with torch.no_grad():` instead.
      img_boxes = Variable(torch.FloatTensor(img_boxes), volatile = True)
    
    
    opened by jinfagang 4
  • None of the links to the Face Antispoofing Datasets are valid.

    None of the links to the Face Antispoofing Datasets are valid.

    I'm on a deadline and i need a descent face anti-spoofing dataset such as Replay-Attack or Casia-FASD. However, none of the links to these datasets are working. Any help?

    opened by debse 3
  • IR-152 wrong password

    IR-152 wrong password

    I'm unable to download IR-152 google auto translate says wrong code used y6ya as written Screenshot from 2019-07-31 14-37-41 I'm using chrome on ubuntu 18.04

    Adding option for any english service such as google drive, onedrive, dropbox etc would be highly appreciated. If download quota for the user is already used, anyone could link it to their own account and download. Approach used in StyleGan models. https://www.geekrar.com/fix-bypass-google-drive-download-limit-error/

    opened by bobiblazeski 3
  • Missing HEAD checkpoint file

    Missing HEAD checkpoint file

    Neither the link to Google Drive nor the link to Baidu contain the .pth file corresponding to the HEAD checkpoints (IR-50 model). Where can I get it? Thanks!

    opened by ealmenzar 0
  • Private Asia Face Data

    Private Asia Face Data

    Hi @ZhaoJ9014, Thank your project. I want to train IR50 with "Private Asia Face Data" from scratch. So, can you provide Private Asia Face Data for me? Thank you.

    opened by vtn98 0
  • How to implement the triple loss?

    How to implement the triple loss?

    The README.md wrote:

    This repo provides a comprehensive face recognition library for face related analytics & applications, including face alignment (detection, landmark localization, affine transformation, etc.), data processing (e.g., augmentation, data balancing, normalization, etc.), various backbones (e.g., ResNet, IR, IR-SE, ResNeXt, SE-ResNeXt, DenseNet, LightCNN, MobileNet, ShuffleNet, DPN, etc.), various losses (e.g., Softmax, Focal, Center, SphereFace, CosFace, AmSoftmax, ArcFace, Triplet, etc.)

    However, we found no triple loss in the repo. Could you tell us how to implement the triple loss?

    Thanks.

    opened by GeorgeCarpenter 1
Owner
Zhao Jian
聚焦 Knowledge changes fate.
Zhao Jian
Face-Recognition-Attendence-System - This face recognition Attendence system using Python

Face-Recognition-Attendence-System I have developed this face recognition Attend

Riya Gupta 4 May 10, 2022
Face Library is an open source package for accurate and real-time face detection and recognition

Face Library Face Library is an open source package for accurate and real-time face detection and recognition. The package is built over OpenCV and us

null 52 Nov 9, 2022
Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

HLA-Face: Joint High-Low Adaptation for Low Light Face Detection The official PyTorch implementation for HLA-Face: Joint High-Low Adaptation for Low L

Wenjing Wang 77 Dec 8, 2022
VGGFace2-HQ - A high resolution face dataset for face editing purpose

The first open source high resolution dataset for face swapping!!! A high resolution version of VGGFace2 for academic face editing purpose

Naiyuan Liu 232 Dec 29, 2022
DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

null 52 Dec 30, 2022
A large-scale face dataset for face parsing, recognition, generation and editing.

CelebAMask-HQ [Paper] [Demo] CelebAMask-HQ is a large-scale face image dataset that has 30,000 high-resolution face images selected from the CelebA da

switchnorm 1.7k Dec 26, 2022
Paddle-Adversarial-Toolbox (PAT) is a Python library for Deep Learning Security based on PaddlePaddle.

Paddle-Adversarial-Toolbox Paddle-Adversarial-Toolbox (PAT) is a Python library for Deep Learning Security based on PaddlePaddle. Model Zoo Common FGS

AgentMaker 17 Nov 8, 2022
A PaddlePaddle version of Neural Renderer, refer to its PyTorch version

Neural 3D Mesh Renderer in PadddlePaddle A PaddlePaddle version of Neural Renderer, refer to its PyTorch version Install Run: pip install neural-rende

AgentMaker 13 Jul 12, 2022
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 6.9k Jan 4, 2023
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Website | Documentation | Tutorials | Installation | Release Notes CatBoost is a machine learning method based on gradient boosting over decision tree

CatBoost 5.7k Feb 12, 2021
PPLNN is a Primitive Library for Neural Network is a high-performance deep-learning inference engine for efficient AI inferencing

PPLNN is a Primitive Library for Neural Network is a high-performance deep-learning inference engine for efficient AI inferencing

null 943 Jan 7, 2023
HyperPose is a library for building high-performance custom pose estimation applications.

HyperPose is a library for building high-performance custom pose estimation applications.

TensorLayer Community 1.2k Jan 4, 2023
Bytedance Inc. 2.5k Jan 6, 2023
The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate. Website • Key Features • How To Use • Docs •

Pytorch Lightning 21.1k Jan 1, 2023
The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate. Website • Key Features • How To Use • Docs •

Pytorch Lightning 11.9k Feb 13, 2021
The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate. Website • Key Features • How To Use • Docs •

Pytorch Lightning 21.1k Jan 8, 2023
A deep learning library that makes face recognition efficient and effective

Distributed Arcface Training in Pytorch This is a deep learning library that makes face recognition efficient, and effective, which can train tens of

Sajjad Aemmi 10 Nov 23, 2021
A PyTorch Toolbox for Face Recognition

FaceX-Zoo FaceX-Zoo is a PyTorch toolbox for face recognition. It provides a training module with various supervisory heads and backbones towards stat

JDAI-CV 1.6k Jan 6, 2023
Pretrained Pytorch face detection (MTCNN) and recognition (InceptionResnet) models

Face Recognition Using Pytorch Python 3.7 3.6 3.5 Status This is a repository for Inception Resnet (V1) models in pytorch, pretrained on VGGFace2 and

Tim Esler 3.3k Jan 4, 2023