Deep Illuminator
Deep Illuminator is a data augmentation tool designed for image relighting. It can be used to easily and efficiently generate a wide range of illumination variants of a single image. It has been tested with several datasets and models and has been shown to succesfully improve performance. It has a built in visualizer created with Streamlit to preview how the target image can be relit. This tool has an accompanying paper.
Example Augmentations
Usage
The simplest method to use this tool is through Docker Hub:
docker pull kartvel/deep-illuminator
Visualizer
Once you have the Deep Illuminator image run the following command to launch the visualizer:
docker run -it --rm --gpus all \
-p 8501:8501 --entrypoint streamlit \
kartvel/deep-illuminator run streamlit/streamlit_app.py
You will be able to interact with it on localhost:8501. Note: If you do not have NVIDIA gpu support enabled for docker simply remove the --gpus all option.
Generating Variants
It is possible to quickly generate multiple variants for images contained in a directory by using the following command:
docker run -it --rm --gpus all \ ─╯
-v /path/to/input/images:/app/probe_relighting/originals \
-v /path/to/save/directory:/app/probe_relighting/output \
kartvel/deep-illuminator --[options]
Options
| Option | Values | Description |
|---|---|---|
| mode | ['synthetic', 'mid'] | Selecting the style of probes used as a relighting guide. |
| step | int | Increment for the granularity of relighted images. max mid: 24, max synthetic: 360 |
Buidling Docker image or running without a container
Please read the following for other options: instructions
Benchmarks
Improved performance of R2D2 for MMA@3 on HPatches
| Training Dataset | Overall | Viewpoint | Illumination |
|---|---|---|---|
| COCO - Original | 71.0 | 65.4 | 77.1 |
| COCO - Augmented | 72.2 (+1.7%) | 65.7 (+0.4%) | 79.2 (+2.7%) |
| VIDIT - Original | 66.7 | 60.5 | 73.4 |
| VIDIT - Augmented | 69.2 (+3.8%) | 60.9 (+0.6%) | 78.1 (+6.4%) |
| Aachen - Original | 69.4 | 64.1 | 75.0 |
| Aachen - Augmented | 72.6 (+4.6%) | 66.1 (+3.1%) | 79.6 (+6.1%) |
Improved performance of R2D2 for the Long-Term Visual Localization challenge on Aachen v1.1
| Training Dataset | 0.25m, 2° | 0.5m, 5° | 5m, 10° |
|---|---|---|---|
| COCO - Original | 62.3 | 77.0 | 79.5 |
| COCO - Augmented | 65.4 (+5.0%) | 83.8 (+8.8%) | 92.7 (+16%) |
| VIDIT - Original | 40.8 | 53.4 | 61.3 |
| VIDIT - Augmented | 53.9 (+32%) | 71.2 (+33%) | 83.2(+36%) |
| Aachen - Original | 60.7 | 72.8 | 83.8 |
| Aachen - Augmented | 63.4 (+4.4%) | 81.7 (+12%) | 92.1 (+9.9%) |
Acknowledgment
The developpement of the VAE for the visualizer was made possible by the PyTorch-VAE repository.
Bibtex
If you use this code in your project, please consider citing the following paper:
@misc{chogovadze2021controllable,
title={Controllable Data Augmentation Through Deep Relighting},
author={George Chogovadze and Rémi Pautrat and Marc Pollefeys},
year={2021},
eprint={2110.13996},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
9.6k Dec 31, 2022
283 Jan 4, 2023
33 Jan 4, 2023
4 Nov 8, 2022
116 Jan 3, 2023
88 Dec 28, 2022
6 Jun 27, 2022
62 Dec 21, 2022
45 Dec 8, 2022
53 Nov 9, 2022
368 Dec 26, 2022
49 Dec 22, 2022
40 Dec 12, 2022
141 Dec 14, 2022
114 Dec 10, 2022
24 Dec 17, 2022
4 Feb 13, 2022