InverseRenderNet: Learning single image inverse rendering
!! Check out our new work InverseRenderNet++ paper and code, which improves the inverse rendering results and shadow handling.
This is the implementation of the paper "InverseRenderNet: Learning single image inverse rendering". The model is implemented in tensorflow.
If you use our code, please cite the following paper:
@inproceedings{yu19inverserendernet,
title={InverseRenderNet: Learning single image inverse rendering},
author={Yu, Ye and Smith, William AP},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2019}
}
Evaluation
Dependencies
To run our evaluation code, please create your environment based on following dependencies:
tensorflow 1.12.0
python 3.6
skimage
cv2
numpy
Pretrained model
- Download our pretrained model from: Link
- Unzip the downloaded file
- Make sure the model files are placed in a folder named "irn_model"
Test on demo image
You can perform inverse rendering on random RGB image by our pretrained model. To run the demo code, you need to specify the path to pretrained model, path to RGB image and corresponding mask which masked out sky in the image. The mask can be generated by PSPNet, which you can find on https://github.com/hszhao/PSPNet. Finally inverse rendering results will be saved to the output folder named by your argument.
python3 test_demo.py --model /PATH/TO/irn_model --image demo.jpg --mask demo_mask.jpg --output test_results
Test on IIW
-
IIW dataset should be downloaded firstly from http://opensurfaces.cs.cornell.edu/publications/intrinsic/#download
-
Run testing code where you need to specify the path to model and IIW data:
python3 test_iiw.py --model /PATH/TO/irn_model --iiw /PATH/TO/iiw-dataset
Training
Train from scratch
The training for InverseRenderNet contains two stages: pre-train and self-train.
- To begin with pre-train stage, you need to use training command specifying option
-mtopre-train. - After finishing pre-train stage, you can run self-train by specifying option
-mtoself-train.
In addition, you can control the size of batch in training, and the path to training data should be specified.
An example for training command:
python3 train.py -n 2 -p Data -m pre-train
Data for training
To directly use our code for training, you need to pre-process the training data to match the data format as shown in examples in Data folder.
In particular, we pre-process the data before training, such that five images with great overlaps are bundled up into one mini-batch, and images are resized and cropped to a shape of 200 * 200 pixels. Along with input images associated depth maps, camera parameters, sky masks and normal maps are stored in the same mini-batch. For efficiency, every mini-batch containing all training elements for 5 involved images are saved as a pickle file. While training the data feeding thread directly load each mini-batch from corresponding pickle file.
101 Dec 12, 2022
354 Jan 1, 2023
14 Dec 29, 2022
34 Nov 28, 2022
120 Jan 5, 2023
417 Oct 3, 2022
84 Jan 5, 2022
496 Jan 5, 2023
215 Dec 7, 2022
930 Jan 4, 2023
24 Apr 28, 2022
686 Dec 26, 2022
127 Dec 3, 2022
213 Dec 26, 2022
4 Mar 19, 2022
1 Nov 5, 2021
13 Dec 17, 2022
1 Jan 27, 2022