Hello,

I tried to run the eval.py using the evaluation dataset (MS1M-ArcFace), but the results for the MixFaceNet-S are suboptimal...

Log file: Training: 2021-09-16 10:52:06,829-[lfw][295672]XNorm: 289.635608 Training: 2021-09-16 10:52:06,837-[lfw][295672]Accuracy-Flip: 0.58800+-0.01770 Training: 2021-09-16 10:52:06,837-[lfw][295672]Accuracy-Highest: 0.58800 Training: 2021-09-16 10:55:39,481-[cfp_fp][295672]XNorm: 2203.600646 Training: 2021-09-16 10:55:39,481-[cfp_fp][295672]Accuracy-Flip: 0.50171+-0.01464 Training: 2021-09-16 10:55:39,481-[cfp_fp][295672]Accuracy-Highest: 0.50171 Training: 2021-09-16 10:58:48,704-[agedb_30][295672]XNorm: 434.067380 Training: 2021-09-16 10:58:48,705-[agedb_30][295672]Accuracy-Flip: 0.49850+-0.00444 Training: 2021-09-16 10:58:48,705-[agedb_30][295672]Accuracy-Highest: 0.49850

I used the '295672backbone.pth' in the output folder.

Thanks for the help in advance!

Hello,

First of all, thank you for sharing your code. I ran into a few problems when trying to run your eval.py code, it would seem like the value of config.gdw_size under the condition if (config.net_size == "s"): should be 1024 instead of 512, without which the model will crash when loaded.

That being fixed, I tried running the model on some of my images and I'm running into some pretty underwhelming results. I haven't tried reproducing your results with existing datasets, but I have no doubt your reported metrics are accurate, so I must be missing a step on my end. This is what I have tried so far

import logging
import os
import sys

import torch

import backbones.mixnetm as mx
from utils.utils_callbacks import CallBackVerification
from utils.utils_logging import init_logging
import numpy as np
from scipy.spatial.distance import cdist

sys.path.append('/root/xy/work_dir/xyface/')
from config import config as cfg
from cv2 import imread, resize


if __name__ == "__main__":
    img1 = imread("o1.jpg")
    img2 = imread("o2.jpg")
    img1 = resize(img1, (112, 112)) / 255
    img2 = resize(img2, (112, 112)) / 255
    img1 = (img1 - 0.5) / 0.5
    img2 = (img2 - 0.5) / 0.5
    backbone = mx.mixnet_s(embedding_size=cfg.embedding_size, width_scale=cfg.scale, gdw_size=cfg.gdw_size).to("cpu")
    backbone.load_state_dict(torch.load(os.path.join('147836backbone.pth'), map_location=torch.device('cpu')))
    model = torch.nn.DataParallel(backbone)
    model.eval()
    img1_embeddings = model(
        torch.tensor(np.expand_dims(np.moveaxis(img1, -1, 0), axis=0)).type(torch.float)).detach().cpu().numpy()
    img2_embeddings = model(
        torch.tensor(np.expand_dims(np.moveaxis(img2, -1, 0), axis=0)).type(torch.float)).detach().cpu().numpy()
    cos_sim = 1 - cdist(img1_embeddings, img2_embeddings, 'cosine')

o1.jpg and o2.jpg are faces detected and aligned by MTCNN, as you did in your paper. As you can see, I reduce the RGB values to the [0,1] range and normalize them with mean and std 0.5, as I saw you did elsewhere in your code. I am not sure what is missing, could you provide me with some assistance, please?

You can find an example of two faces below, I get the cos_sim 0.92102719 when comparing them, which seems too high. https://drive.google.com/drive/folders/1YUY3ZkxPQFemWzQ8byHdNFUidxw5OOKp?usp=sharing