The source code of CVPR17 'Generative Face Completion'.

Overview

GenerativeFaceCompletion

Matcaffe implementation of our CVPR17 paper on face completion.

In each panel from left to right: original face, masked input, completion result.

Setup

  • We use the caffe-for-cudnn-v2.5.48. Please refer Caffe for more installation details.
  • Basically, you need to first modify the MATLAB_DIR in Makefile.config and then run the following commands for a successful compilation:
make all -j4
make matcaffe

Training

  • Follow the DCGAN to prepare the data (CelebA). The only differece is that the face we cropped is of size 128x128. Please modify Line 10 in their crop_celebA.lua file. We use the standard train&test split of the CelebA dataset.

  • Modify the training data path in ./matlab/FaceCompletion_training/GFC_caffeinit.m file.

  • Download our face parsing model Model_parsing and put it under ./matlab/FaceCompletion_training/model/ folder.

  • We provide an initial model that is only trained with the reconstruction loss, as a good start point for the subsequent GAN training. Please download it and put it under ./matlab/FaceCompletion_training/model/ folder.

  • Run ./matlab/FaceCompletion_training/demo_GFC_training.m for training.

Testing

  • Download our face completion model Model_G and put it under ./matlab/FaceCompletion_testing/model/ folder.
  • Run ./matlab/FaceCompletion_testing/demo_face128.m for completion. TestImages are from the CelebA test dataset.

Citation

@inproceedings{GFC-CVPR-2017,
    author = {Li, Yijun and Liu, Sifei and Yang, Jimei and Yang, Ming-Hsuan},
    title = {Generative Face Completion},
    booktitle = {IEEE Conference on Computer Vision and Pattern Recognition},
    year = {2017}
}

Acknowledgement

  • Gratitude goes to Sifei Liu for the great help on code.
  • The upsample layer (unpooling according to the pooling mask) is borrowed from the SegNet.
Comments
  • Caffe doesn't declare its version in headers

    Caffe doesn't declare its version in headers

    @Yijunmaverick Hi !See your item , i'm very intresting ! But , i can't build it ! Because of the caffe version is not confirm!!! at Setup Introduction We use the caffe version v2.5.48. Please refer Caffe for more installation details ! What's means caffe v2.5.48 ?????

    opened by lensea 9
  • a problem about demo_face128.m

    a problem about demo_face128.m

    I have encountered a problem when I ran the demo_face128.m. Error occurs at line 36: net = caffe.Net(net_model, net_weights, phase); It shows that: Error parsing text-format caffe.NetParameter: 372:18: Message type "caffe.LayerParameter" has no field named "upsample_param". Is that I failed to compile caffe? but I run make matcaffe and mattest successfully Do you know how to solve it? Thank you!

    opened by amy6chunxiao 4
  • The parsing network does not give desired output

    The parsing network does not give desired output

    Hi @Yijunmaverick ,

    When I was trying to run your parsing demo, I can not get it work correctly. I implemented your demo in python by following the matlab code step by step. However, the output (as shown below) does not give a good segmentation result. Could you please help me to point out what is wrong with the following code? Or maybe the .caffemodel file itself is not the correct one?

    Thanks, Haofu

    import caffe
    import numpy as np
    from PIL import Image
    import matplotlib.pyplot as plt
    
    model_def = "Model_parsing.prototxt"
    model_weights = "Model_parsing.caffemodel"
    net = caffe.Net(model_def, model_weights, caffe.TEST)
    caffe.set_mode_cpu()
    
    image_file = "../../repos/gfc/matlab/FaceCompletion_testing/TestImages/182701.png"
    image = np.array(Image.open(image_file))
    
    # preprocessing the image to fit the net requirement
    input_ = image / 255.0
    input_ = input_ * 2 - 1
    input_ = input_.transpose(2, 0, 1)
    input_ = input_[np.newaxis, ...]
    
    net.blobs['data'].reshape(*input_.shape)
    net.blobs['data'].data[...] = input_
    output = net.forward()
    scores = output['conv_decode0'][0]
    segmentation = scores.argmax(0)
    segmentation_rgb = np.zeros(image.shape, dtype=np.uint8)
    
    colors = [
        [0, 0, 255],
        [255, 255, 0],
        [160, 32, 240],
        [218, 112, 214],
        [210, 105, 30],
        [94, 38, 18],
        [0, 255, 0],
        [156, 102, 31],
        [0, 0, 0],
        [255, 127, 80],
        [255, 0, 0]
    ]
    
    for i in range(11):
        segmentation_rgb[np.where(segmentation == i)] = colors[i]
    
    plt.figure(1)
    plt.subplot(121)
    plt.imshow(image)
    plt.subplot(122)
    plt.imshow(segmentation_rgb)
    

    image

    opened by liaohaofu 3
  • Bad image completion with pretrained model

    Bad image completion with pretrained model

    Hi,

    I tried testing your pretrained Model_G provided here. I tested a sample image that comes in the ./matlab/FaceCompletion_testing/TestImages/ folder with your pretrained model using the ./matlab/FaceCompletion_testing/demo_face128.m file. However, the completion results looks bad. It just seems like a brown plain color and no face completion.

    image

    Also, in the demo_face128.m file, there seemed to have been a bug where line 63 was scores = scores{1} which was a 64x64x32 tensor. I changed it to scores = scores{5} which infact was a 128x128x3 sized tensor. Can you tell me what's going wrong here. Thanks.

    opened by rahuldey91 0
  • Specific version that support to build matcaffe

    Specific version that support to build matcaffe

    Hello, im wondering what is the combination of Ubuntu, MATLAB, gcc, g++, cuda and cudnn to build matcaffe?

    I have Ubuntu 16.04 with gcc 5.4, CUDA 9.2 and cudnn 7.5. Works well with caffe make test but not with make matcaffe. Matlab 2015a - 2016a need gcc 4.7 Matlab 2016b - 2017b need gcc 4.9 matlab 2018a - 2019b need gcc 6.3

    opened by avinmaulana 0
  • question about  the fully-connected layer

    question about the fully-connected layer

    Hello, I have read this paper. I'd like to use this network to process images much larger than 128X128 and the fully-connected layers (FC) will limit the dimension of input,so I wonder that what if you drop that two FC layers in Generator? If you have conducted such experiments, could you please share the results to me. Thanks

    opened by xueweili 0
  • caffe.solver problem

    caffe.solver problem

    @Yijunmaverick when i ran the demo.training.m for training , the matlab showed these errors,: Error in caffe.get_solver (line 7) pSolver = caffe_('get_solver', solver_file);

    Error in caffe.Solver (line 22) self = caffe.get_solver(varargin{:});

    Error in GFC_caffeinit (line 6) Solver_ = caffe.Solver(solver_file); % c++

    Error in demo_GFC_training (line 22) [Solver_G_, Solver_G] = GFC_caffeinit(solver_file_G, save_file_G);

    but i ran make matcaffe successfully, have you encountered this problem before?

    opened by xinyao123 0
  • a problem while running the demo_GFC_training.m

    a problem while running the demo_GFC_training.m

    @Yijunmaverick while running the demo, the matlab shows a error like this:

    Error using caffe_ Could not open file model/Model_G_solver.prototxt

    is it my path?

    opened by Yuanyuan0218 0
Owner
Yijun Li
人生自是有情痴,此恨不关风与月
Yijun Li
Face Library is an open source package for accurate and real-time face detection and recognition

Face Library Face Library is an open source package for accurate and real-time face detection and recognition. The package is built over OpenCV and us

null 52 Nov 9, 2022
Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

HLA-Face: Joint High-Low Adaptation for Low Light Face Detection The official PyTorch implementation for HLA-Face: Joint High-Low Adaptation for Low L

Wenjing Wang 77 Dec 8, 2022
img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation

img2pose: Face Alignment and Detection via 6DoF, Face Pose Estimation Figure 1: We estimate the 6DoF rigid transformation of a 3D face (rendered in si

Vítor Albiero 519 Dec 29, 2022
DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

null 52 Dec 30, 2022
[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

SADRNet Paper link: SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction Requirements python

Multimedia Computing Group, Nanjing University 99 Dec 30, 2022
Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection ?? Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

Prem Kumar 86 Aug 3, 2022
Swapping face using Face Mesh with TensorFlow Lite

Swapping face using Face Mesh with TensorFlow Lite

iwatake 17 Apr 26, 2022
Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels.

The Face Synthetics dataset Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels. It was introduced in ou

Microsoft 608 Jan 2, 2023
VGGFace2-HQ - A high resolution face dataset for face editing purpose

The first open source high resolution dataset for face swapping!!! A high resolution version of VGGFace2 for academic face editing purpose

Naiyuan Liu 232 Dec 29, 2022
A large-scale face dataset for face parsing, recognition, generation and editing.

CelebAMask-HQ [Paper] [Demo] CelebAMask-HQ is a large-scale face image dataset that has 30,000 high-resolution face images selected from the CelebA da

switchnorm 1.7k Dec 26, 2022
Python tools for 3D face: 3DMM, Mesh processing(transform, camera, light, render), 3D face representations.

face3d: Python tools for processing 3D face Introduction This project implements some basic functions related to 3D faces. You can use this to process

Yao Feng 2.3k Dec 30, 2022
Face-Recognition-Attendence-System - This face recognition Attendence system using Python

Face-Recognition-Attendence-System I have developed this face recognition Attend

Riya Gupta 4 May 10, 2022
AI Face Mesh: This is a simple face mesh detection program based on Artificial intelligence.

AI Face Mesh: This is a simple face mesh detection program based on Artificial Intelligence which made with Python. It's able to detect 468 different

Md. Rakibul Islam 1 Jan 13, 2022
Video-face-extractor - Video face extractor with Python

Python face extractor Setup Create the srcvideos and faces directories Put your

null 2 Feb 3, 2022
Face and Pose detector that emits MQTT events when a face or human body is detected and not detected.

Face Detect MQTT Face or Pose detector that emits MQTT events when a face or human body is detected and not detected. I built this as an alternative t

Jacob Morris 38 Oct 21, 2022
Implementation for Paper "Inverting Generative Adversarial Renderer for Face Reconstruction"

StyleGAR TODO: add arxiv link Implementation of Inverting Generative Adversarial Renderer for Face Reconstruction TODO: for test Currently, some model

null 155 Oct 27, 2022
A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.

AnimeGAN A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing. Randomly Generated Images The images are

Jie Lei 雷杰 1.2k Jan 3, 2023
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Thomas Neumann 117 Nov 27, 2022
Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

Code for EmBERT, a transformer model for embodied, language-guided visual task completion.

null 41 Jan 3, 2023