Self-supervised Outdoor Scene Relighting
This is the implementation of the paper "Self-supervised Outdoor Scene Relighting". The model is implemented in tensorflow.
If you use our code, please cite the following paper:
@inproceedings{yu20relightNet,
title={Self-supervised Outdoor Scene Relighting},
author={Yu, Ye and Meka, Abhimitra and Elgharib, Mohamed and Seidel, Hans-Peter and Theobalt, Christian and Smith, William A. P.},
booktitle={Proc. of the European Conference on Computer Vision (ECCV)},
year={2020}
}
Evaluation
Dependencies
To run our evaluation code, please create your environment based on following dependencies:
tensorflow 1.12.0
python 3.6
skimage
cv2
PIL
numpy
Pretrained model
Relighting model
- Download our pretrained relighting model from: Link
- Make sure the model files are placed in a folder named "relight_model"
Sky model
- Download our pretrained sky generation model from: Link
- Make sure the model files are placed in a folder named "model_skyGen_net"
Test on time-lapse image pair
The following code performs relighting on a pair of demo time-lapse images, which are placed in "timeLapse_imgs" folders. It uses our pre-trained model to relight "2.jpg" by borrowing the illumination from "1.jpg". The provided mask map can be generated by PSPNet, which you can find on https://github.com/hszhao/PSPNet. The relighting image is saved as timeLapse_rendering.png.
python3 test_timelpase.py
Relight image by time-lapse video
We demonstrate our performance by relighting an input image under illuminations captured by a time-lapse video. Inputs are stored in "timeLapse_illu". Again the mask file can be generated by PSPNet.
python3 test_timelpase_illu.py