Hello, thank your for sharing this amazing work.

When I run the python singleImage.py -f img command, I got the error that:

Process the image:  img/sample_1.jpg
Traceback (most recent call last):
  File "singleImage.py", line 129, in <module>
    main(args)
  File "singleImage.py", line 106, in main
    render(img_ori, vertices_lst, alpha=0.6, wfp=f'inference_output/rendering_overlay/{name}.jpg')
  File "~/SynergyNet/utils/render.py", line 42, in render
    overlap = render_app(ver, tri, overlap, texture=tex)
UnboundLocalError: local variable 'tri' referenced before assignment

I found that it is because the connectivity argument is None by default, but I don't know how to set it to a correct value.

When I load file './3dmm_data/param_all_norm_v201.pkl' , I found that it has shape ([636252, 102]). I guess that 636252 is the number of image for training but I still dont understand that what 102 is? Can you explain to me about how you can create your training set?

It seems that some hyper-parameter settings in the code (train_script.sh) are inconsistent with those in the paper, for example, learning rate (0.027 vs. 0.08), loss weight \lambda_{2} (0.05 vs. 0.03), batch size (900 vs. 1024), milestones (48 & 64 vs. 30 & 40), epoch number (50 vs. 80) and lr decay (0.2 vs. 0.1). Of course these numbers are adjustable but important in the experiments. I want to know how to set these hyper-parameters with backbone mobilenetv2 to get a good performance as yours.

Hello, I'd been reading and executing your code and noticed that you are using ToTensor transform from torchvision. I think you need to be aware of it's scaling effects. Lets break down transform = transforms.Compose([ToTensor(), Normalize(mean=127.5, std=128)]) Assume the intensity of a pixel is 128. ToTensor converts it to float, BUT ALSO divides it by 255. So now your intensity is 0.5 Then in Normalize: (0.5 - 127.5) / 128 = -127 / 128 ~ -0.97. Applying it to other values from range withing [0, 255], we get a range for your intensity in [-0.998, 0.996], which is quiet opposite for what I would consider normalization

The fix is pretty simple, that is to use PILToTensor from the same torchvision, that does not scale your input by 255. Doint that you will get way normal normalization as shown below, in (-1, 1) :)

I will try retraining and benchmarking with better normalization and will submit a pr

Additional python requirements that are needed, but not listed, are: cython and torchvision

Also, and more importantly, this project requires a cuda compatible GPU to run.

Hi, when I try to run python benchmark.py -w pretrained/best.pth.tar, I encounter such an error:

(SynergyNet) heyuan@VIML4:~/Research/3d_face/SynergyNet$ python benchmark.py -w pretrained/best.pth.tar
Traceback (most recent call last):
  File "benchmark.py", line 256, in <module>
    main()
  File "benchmark.py", line 252, in main
    benchmark(args.weights, args)
  File "benchmark.py", line 241, in benchmark
    aflw2000()
  File "benchmark.py", line 235, in aflw2000
    batch_size=128)
  File "benchmark.py", line 113, in extract_param
    model.load_state_dict(checkpoint)
  File "/home/heyuan/Environments/anaconda3/envs/deep3d_pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1045, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DataParallel:
	Unexpected key(s) in state_dict: "module.u_tex", "module.w_tex". 

Do you have any idea to solve it?

Hi, Thanks for sharing your amazing code!

I am trying to run your training code. In Training 2, you mentioned "Download training data from [3DDFA]: train_aug_120x120.zip and extract the zip file under the root folder (This training size is about 44G)."

However, when I open 3DDFA, I only find a train_aug_120x120.zip with 2.15G. After extracting, it only has 2.8G, which is far less than 44G.

I am wondering, am I missing some information to get the whole dataset?

Hi,

I hope you are doing well. I wanted to ask, what is the camera intrinsic that you consider while projecting the mesh and landmarks back to the 2D image?

I checked the performance,but do it can not meet the target(1.27 | 1.59 | 1.31) in NoW benchmark using the given pretrained checkpoint. Thanks very much.

Hi, I am YJHong and thanks for sharing great work!

I checked code measuring landmark alignment (NME, benchmark_alfw2000.py) though, couldn't find any related code for measuring face orientation (pitch/yaw/roll).

Would you let me know how measure face orientation given 3d landmarks ? (or any related code / repo)

• Have you used cv2.solvePnP function for estimating euler angle ?

Could you provide the evaluation code of AFLW dataset?

Hi, thank your for sharing this amazing work.

When I ran the demo inference for the in-the-wild images, I got results they are not good.

The input image:

The landmark image:

The blended image:

You can easily find the results have misalignment, I am not sure whether I did something wrong.

BTW, the image size is 512x512.