Code for Deep Single-image Portrait Image Relighting

Related tags

Deep Learning DPR
Overview

Deep Single-Image Portrait Relighting [Project Page]

Hao Zhou, Sunil Hadap, Kalyan Sunkavalli, David W. Jacobs. In ICCV, 2019

Overview

  • Test script for 512x512 images: testNetwork_demo_512.py
  • Test script for 1024x1024 images: testNetwork_demo_1024.py

Dependencies

pytorch >= 1.0.0

opencv >= 4.0.0

shtools: https://shtools.oca.eu/shtools/ (optional)

Notes

We include an example image and seven example lightings in data. Note that different methods may have different coordinate system for Spherical Harmonics (SH), you may need to change the coordiante system if you use SH lighting from other sources. The coordinate system of our method is in accordance with shtools, we provide a function utils_normal.py in utils to help you tansfer the coordinate system from bip2017 and sfsNet to our coordinate system. To use utils_normal.py you need to install shtools. The code is for research purpose only.

Data Preparation

We publish the code for data preparation, please find it in (https://github.com/zhhoper/RI_render_DPR).

Citation

If you use this code for your research, please consider citing:

@InProceedings{DPR,
  title={Deep Single Portrait Image Relighting},
  author = {Hao Zhou and Sunil Hadap and Kalyan Sunkavalli and David W. Jacobs},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2019}
}
Comments
  • How To Generate Custom 'rotate_light_00.txt' File

    How To Generate Custom 'rotate_light_00.txt' File

    I'm trying to understand how the values in these files were generated.

    Am I right to assume that the 9 values in each rotate_light_xx.txt file are the first nine spherical harmonic coefficients?

    Any advice on how someone could define new ones based on a new direction the light is coming from?

    opened by JeremyOgo 2
  • Problems to download the dataset

    Problems to download the dataset

    Hi, your work is amazing and thanks to share with us.

    But I have a problem to download the dataset, Google say "download quota exceeded". Do you know about that? Do you have other methods to download the dataset?

    opened by naanadr 2
  • How do I get file like“rotate_light_00.txt” file from a face image?

    How do I get file like“rotate_light_00.txt” file from a face image?

    Thanks for your excellent work! I have a question, how can i get file like“rotate_light_00.txt” file from a face image? I want to do some illumination transfer work. thank you!

    opened by cccml 0
  • outputSH only works for the test image obama

    outputSH only works for the test image obama

    Hi, I was trying to test the code on this repo with some custom face images but the outputSH is always all black (zeros) or always all white (in the circle). Isn't the algorithm supposed to compute the lighting of the original input image correctly? Or there is something that we need to change or use to do so?

    opened by fsbarros98 0
  • Create video with relighting effect from single image

    Create video with relighting effect from single image

    Thanks for your research. It's work extremely very well. I have some contributes to make it better. Description below:

    Feature:

    • Create video with relighting effect from single image

    Refactor:

    • Easy implement code with DPR class
    • Easy adjustment configs with .yaml file
    • Inference model 512 or 1024 in one script

    Result


    Video rendered from single image with difference rotate light

    Question

    I readed all issue in repo and your code in utils, but i can't generate lighting with coordinate i want. Can you write example code demo for this?

    opened by KudoKhang 0
  • How to convert sfsNet SH to shtools？

    How to convert sfsNet SH to shtools？

    Hello, I extracted 27 dimension SH coefficients from sfsNet ( 9 dimension for each channel of RGB), I see you provide the "convert code", but the output dimension is still 27, your network input is 9 dimension. Can you show me a code example of how to convert？Thank you so much！！！！！！

    opened by cccml 3
  • How can I normalize predicted SH light to a score to evaluate brightness?

    How can I normalize predicted SH light to a score to evaluate brightness?

    Hi @zhhoper, Thank you for your work, I am working about evaluate brightness of face image and I want to compare a predicted SH light and label SH light to identify bad brightness image and good brightness image. Can you tell me how to do it? Thanks.

    opened by chuan298 0
  • Question About Test Dataset

    Question About Test Dataset

    Hello, thanks for your wonderful work. Could you please send me the test dataset for quantitative results in your paper? Or, how can I download the test part of the Multi-PIE dataset? Thanks.

    opened by Seanseattle 1
  • What is the ori_shading.exr?

    What is the ori_shading.exr?

    Thank you for your sharing!

    Well, I have a question about the DPR_dataset. There is a file named ori_shading.exr, and I cannot find it in RI_render_DPR's result. So how to make the ori_shading.exr, is this the same file with ori_shading.npy?

    opened by changbozhu 1
Owner
null
GitHub https://zhhoper.github.io/dpr.html
Code for PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Relighting and Material Editing

PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Relighting and Material Editing CVPR 2021. Project page: https://kai-46.github.io/

Kai Zhang 141 Dec 14, 2022
The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

Ren Yurui 261 Jan 9, 2023
The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

Website | ArXiv | Get Start | Video PIRenderer The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic

Ren Yurui 81 Sep 25, 2021
《Towards High Fidelity Face Relighting with Realistic Shadows》(CVPR 2021)

Towards High Fidelity Face-Relighting with Realistic Shadows Andrew Hou, Ze Zhang, Michel Sarkis, Ning Bi, Yiying Tong, Xiaoming Liu. In CVPR, 2021. T

null 114 Dec 10, 2022
This is the implementation of the paper "Self-supervised Outdoor Scene Relighting"

Self-supervised Outdoor Scene Relighting This is the implementation of the paper "Self-supervised Outdoor Scene Relighting". The model is implemented

Ye Yu 24 Dec 17, 2022
Official code for paper Exemplar Based 3D Portrait Stylization.

3D-Portrait-Stylization This is the official code for the paper "Exemplar Based 3D Portrait Stylization". You can check the paper on our project websi

null 60 Dec 7, 2022
[TOG 2021] PyTorch implementation for the paper: SofGAN: A Portrait Image Generator with Dynamic Styling.

This repository contains the official PyTorch implementation for the paper: SofGAN: A Portrait Image Generator with Dynamic Styling. We propose a SofGAN image generator to decouple the latent space of portraits into two subspaces: a geometry space and a texture space. Experiments on SofGAN show that our system can generate high quality portrait images with independently controllable geometry and texture attributes.

Anpei Chen 694 Dec 23, 2022
dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ)

dualFace dualFace: Two-Stage Drawing Guidance for Freehand Portrait Sketching (CVMJ) We provide python implementations for our CVM 2021 paper "dualFac

Haoran XIE 46 Nov 10, 2022
Official Implementation and Dataset of "PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency", CVPR 2021

Portrait Photo Retouching with PPR10K Paper | Supplementary Material PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask an

null 184 Dec 11, 2022
MODNet: Trimap-Free Portrait Matting in Real Time

MODNet is a model for real-time portrait matting with only RGB image input.

Zhanghan Ke 2.8k Dec 30, 2022
Rethinking Portrait Matting with Privacy Preserving

Rethinking Portrait Matting with Privacy Preserving This is the official repository of the paper Rethinking Portrait Matting with Privacy Preserving.

null 184 Jan 3, 2023
Code associated with the paper "Deep Optics for Single-shot High-dynamic-range Imaging"

Deep Optics for Single-shot High-dynamic-range Imaging Code associated with the paper "Deep Optics for Single-shot High-dynamic-range Imaging" CVPR, 2

Stanford Computational Imaging Lab 40 Dec 12, 2022
PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

About PyTorch 1.2.0 Now the master branch supports PyTorch 1.2.0 by default. Due to the serious version problem (especially torch.utils.data.dataloade

Sanghyun Son 2.1k Jan 1, 2023
Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution"

NTIRE2017 Super-resolution Challenge: SNU_CVLab Introduction This is our project repository for CVPR 2017 Workshop (2nd NTIRE). We, Team SNU_CVLab, (B

Bee Lim 625 Dec 30, 2022
RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020)

RCDNet: A Model-driven Deep Neural Network for Single Image Rain Removal (CVPR2020) Hong Wang, Qi Xie, Qian Zhao, and Deyu Meng [PDF] [Supplementary M

Hong Wang 6 Sep 27, 2022
Official PyTorch code of Holistic 3D Scene Understanding from a Single Image with Implicit Representation (CVPR 2021)

Implicit3DUnderstanding (Im3D) [Project Page] Holistic 3D Scene Understanding from a Single Image with Implicit Representation Cheng Zhang, Zhaopeng C

Cheng Zhang 149 Jan 8, 2023
PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

HAN PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network" This repository is for HAN introduced in the

五维空间 140 Nov 23, 2022
Code for generating a single image pretraining dataset

Single Image Pretraining of Visual Representations As shown in the paper A critical analysis of self-supervision, or what we can learn from a single i

Yuki M. Asano 12 Dec 19, 2022
Code for 'Single Image 3D Shape Retrieval via Cross-Modal Instance and Category Contrastive Learning', ICCV 2021

CMIC-Retrieval Code for Single Image 3D Shape Retrieval via Cross-Modal Instance and Category Contrastive Learning. ICCV 2021. Introduction In this wo

null 42 Nov 17, 2022