face3d: Python tools for processing 3D face
Introduction
This project implements some basic functions related to 3D faces.
You can use this to process mesh data, generate 3D faces from morphable model, reconstruct 3D face with a single image and key points as inputs, render faces with difference lightings(for more, please see examples).
In the beginning, I wrote this project for learning 3D face reconstruction and for personal research use, so all the codes are written in python(numpy). However, some functions(eg. rasterization) can not use vectorization to optimize, writing them in python is too slow to use, then I choose to write these core parts in c++(without any other big libraries, such as opencv, eigen) and compile them with Cython for python use. So the final version is very lightweight and fast.
In addition, the numpy version is also retained, considering that beginners can focus on algorithms themselves in python and researches can modify and verify their ideas quickly. I also try my best to add references/formulas in each function, so that you can learn basic knowledge and understand the codes.
For more information and researches related to 3D faces, please see 3D face papers.
Enjoy it ^_^
Structure
# Since triangle mesh is the most popular representation of 3D face,
# the main part is mesh processing.
mesh/ # written in python and c++
| cython/ # c++ files, use cython to compile
| io.py # read & write obj
| vis.py # plot mesh
| transform.py # transform mesh & estimate matrix
| light.py # add light & estimate light(to do)
| render.py # obj to image using rasterization render
mesh_numpy/ # the same with mesh/, with each part written in numpy
# slow but easy to learn and modify
# 3DMM is one of the most popular methods to generate & reconstruct 3D face.
morphable_model/
| morphable_model.py # morphable model class: generate & fit
| fit.py # estimate shape&expression parameters. 3dmm fitting.
| load.py # load 3dmm data
Examples:
cd ./examples
-
3dmm.
python 2_3dmm.pyleft: random example generated by 3dmm
right: fitting face with 3dmm using 68 key points
-
transform.
python 3_transform.py
left:fix camera position & use orthographic projection. (often used in reconstruction)
then transform face object: scale, change pitch angle, change yaw angle, change roll angle
right:
fix obj position & use perspective projection(fovy=30). (simulating real views)
then move camera position and rotate camera: from far to near, down & up, left & right, rotate camera
-
light.
python 4_light.pysingle point light: from left to right, from up to down, from near to far
-
image map
python 6_image_map.pyrender different attributes in image pixels.
: depth, pncc, uv coordinates
-
uv map
python 7_uv_map.pyrender different attributes in uv coordinates.
: colors(texture map), position(2d facial image & corresponding position map)
Getting Started
Prerequisite
-
Python 2 or Python 3
-
Python packages:
- numpy
- skimage (for reading&writing image)
- scipy (for loading mat)
- matplotlib (for show)
- Cython (for compiling c++ file)
Usage
-
Clone the repository
git clone https://github.com/YadiraF/face3d cd face3d -
Compile c++ files to .so for python use (ignore if you use numpy version)
cd face3d/mesh/cython python setup.py build_ext -i -
Prepare BFM Data (ignore if you don't use 3dmm)
-
Run examples
(examples use cython version, you can change
meshintomesh_numpyto use numpy version)cd examples python 1_pipeline.pyFor beginners who want to continue researches on 3D faces, I strongly recommend you first run examples according to the order, then view the codes in mesh_numpy and read the comments written in the beginning in each file. Hope this helps!
Moreover, I am new in computer graphics, so it would be great appreciated if you could point out some of my wrong expressions. Thanks!
Changelog
- 2018/10/08 change structure. add comments. add introduction. add paper collections.
- 2018/07/15 first release
1 Jan 13, 2022
305 Dec 16, 2022
3 Feb 15, 2022
53 Dec 13, 2022
251 Jan 8, 2023
10 Oct 11, 2022
2 Dec 24, 2021
17 Apr 26, 2022
77 Dec 8, 2022
25 Dec 26, 2022
494 Jan 6, 2023
221 Dec 30, 2022
217 Dec 12, 2022
51 Nov 22, 2022
42 Nov 16, 2022
1.3k Dec 25, 2022
715 Nov 5, 2022
212 Dec 29, 2022
6 Dec 5, 2022