Lighting the Darkness in the Deep Learning Era: A Survey, An Online Platform, A New Dataset

Related tags

Deep Learning Lighting-the-Darkness-in-the-Deep-Learning-Era-Open
Overview

Lighting the Darkness in the Deep Learning Era: A Survey, An Online Platform, A New Dataset

This repository provides a unified online platform, LoLi-Platform http://mc.nankai.edu.cn/ll/, that covers many popular deep learning-based LLIE methods, of which the results can be produced through a user-friendly web interface, contains a low-light image and video dataset, LoLi-Phone (will be released soon), in which the images and videos are taken by various phones' cameras under diverse illumination conditions and scenes, and collects deep learning-based low-light image and video enhancement methods, datasets, and evaluation metrics. More content and details can be found in our Survey Paper: Lighting the Darkness in the Deep Learning Era. We provide the comparison results on the real low-light videos taken by different mobile phones’ cameras at YouTube https://www.youtube.com/watch?v=Elo9TkrG5Oo&t=6s.

✌ We will periodically update the content. Welcome to let us know if we miss your work that is published in top-tier Journal or conference. We will add it.

✌ Our LoLi-Platform supports the function of download. Please right click and then save the figure.

✌ If you use this dataset or platform, please cite our paper. Please hit the star at the top-right corner. Thanks!

Contents

  1. LoLi-Platform
  2. LoLi-Phone Dataset
  3. Methods
  4. Datasets
  5. Metrics
  6. Citation

LoLi-Platform

Currently, the LoLi-Platform covers 13 popular deep learning-based LLIE methods including LLNet, LightenNet, Retinex-Net, EnlightenGAN, MBLLEN, KinD, KinD++, TBEFN, DSLR, DRBN, ExCNet, Zero-DCE, and RRDNet, where the results of any inputs can be produced through a user-friendly web interface. Have fun: LoLi-Platform.

LoLi-Phone

Overview LoLi-Phone dataset contains 120 videos (55,148 images) taken by 18 different phones' cameras including iPhone 6s, iPhone 7, iPhone7 Plus, iPhone8 Plus, iPhone 11, iPhone 11 Pro, iPhone XS, iPhone XR, iPhone SE, Xiaomi Mi 9, Xiaomi Mi Mix 3, Pixel 3, Pixel 4, Oppo R17, Vivo Nex, LG M322, OnePlus 5T, Huawei Mate 20 Pro under diverse illumination conditions (e.g., weak illumination, underexposure, dark, extremely dark, back-lit, non-uniform light, color light sources, etc.) in the indoor and outdoor scenes. Anyone can access the LoLi-Phone dataset.

Methods

Overview

Date Publication Title Abbreviation Code Platform
2017 PR LLNet: A deep autoencoder approach to natural low-light image enhancement paper LLNet Code Theano
2018 PRL LightenNet: A convolutional neural network for weakly illuminated image enhancement paper LightenNet Code Caffe & MATLAB
2018 BMVC Deep retinex decomposition for low-light enhancement paper Retinex-Net Code TensorFlow
2018 BMVC MBLLEN: Low-light image/video enhancement using CNNs paper MBLLEN Code TensorFlow
2018 TIP Learning a deep single image contrast enhancer from multi-exposure images paper SCIE Code Caffe & MATLAB
2018 CVPR Learning to see in the dark paper Chen et al. Code TensorFlow
2018 NeurIPS DeepExposure: Learning to expose photos with asynchronously reinforced adversarial learning paper DeepExposure TensorFlow
2019 ICCV Seeing motion in the dark paper Chen et al. Code TensorFlow
2019 ICCV Learning to see moving object in the dark paper Jiang and Zheng Code TensorFlow
2019 CVPR Underexposed photo enhancement using deep illumination estimation paper DeepUPE Code TensorFlow
2019 ACMMM Kindling the darkness: A practical low-light image enhancer paper KinD Code TensorFlow
2019 ACMMM (IJCV) Kindling the darkness: A practical low-light image enhancer paper (Beyond brightening low-light images paper) KinD (KinD++) Code TensorFlow
2019 ACMMM Progressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement paper Wang et al. Caffe
2019 TIP Low-light image enhancement via a deep hybrid network paper Ren et al. Caffe
2019(2021) arXiv(TIP) EnlightenGAN: Deep light enhancement without paired supervision paper arxiv EnlightenGAN Code PyTorch
2019 ACMMM Zero-shot restoration of back-lit images using deep internal learning paper ExCNet Code PyTorch
2020 CVPR Zero-reference deep curve estimation for low-light image enhancement paper Zero-DCE Code PyTorch
2020 CVPR From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement paper DRBN Code PyTorch
2020 ACMMM Fast enhancement for non-uniform illumination images using light-weight CNNs paper Lv et al. TensorFlow
2020 ACMMM Integrating semantic segmentation and retinex model for low light image enhancement paper Fan et al.
2020 CVPR Learning to restore low-light images via decomposition-and-enhancement paper Xu et al. PyTorch
2020 AAAI EEMEFN: Low-light image enhancement via edge-enhanced multi-exposure fusion network paper EEMEFN PyTorch
2020 TIP Lightening network for low-light image enhancement paper DLN PyTorch
2020 TMM Luminance-aware pyramid network for low-light image enhancement paper LPNet PyTorch
2020 ECCV Low light video enhancement using synthetic data produced with an intermediate domain mapping paper SIDGAN TensorFlow
2020 TMM TBEFN: A two-branch exposure-fusion network for low-light image enhancement paper TBEFN Code TensorFlow
2020 ICME Zero-shot restoration of underexposed images via robust retinex decomposition paper RRDNet Code PyTorch
2020 TMM DSLR: Deep stacked laplacian restorer for low-light image enhancement paper DSLR Code PyTorch

Datasets

Abbreviation Number Format Real/Synetic Video Paired/Unpaired/Application Dataset
LOL paper 500 RGB Real No Paired Dataset
SCIE paper 4413 RGB Real No Paired Dataset
MIT-Adobe FiveK paper 5000 Raw Real No Paired Dataset
SID paper 5094 Raw Real No Paired Dataset
DRV paper 202 Raw Real Yes Paired Dataset
SMOID paper 179 Raw Real Yes Paired Dataset
LIME paper 10 RGB Real No Unpaired Dataset
NPE paper 84 RGB Real No Unpaired Dataset
MEF paper 17 RGB Real No Unpaired Dataset
DICM paper 64 RGB Real No Unpaired Dataset
VV 24 RGB Real No Unpaired Dataset
ExDARK paper 7363 RGB Real No Application Dataset
BBD-100K paper 10,000 RGB Real Yes Application Dataset
DARK FACE paper 6000 RGB Real No Application Dataset

Metrics

Abbreviation Full-/Non-Reference Platform Code
MAE (Mean Absolute Error) Full-Reference
MSE (Mean Square Error) Full-Reference
PSNR (Peak Signal-to-Noise Ratio) Full-Reference
SSIM (Structural Similarity Index Measurement) Full-Reference MATLAB Code
LPIPS (Learned Perceptual Image Patch Similarity) Full-Reference PyTorch Code
LOE (Lightness Order Error) Non-Reference MATLAB Code
NIQE (Naturalness Image Quality Evaluator) Non-Reference MATLAB Code
PI (Perceptual Index) Non-Reference MATLAB Code
SPAQ (Smartphone Photography Attribute and Quality) Non-Reference PyTorch Code
NIMA (Neural Image Assessment) Non-Reference PyTorch/TensorFlow Code/Code

Citation

If you find the repository helpful in your resarch, please cite the following paper.

@article{LoLi,
  title={Lighting the Darkness in the Deep Learning Era},
  author={Li, Chongyi and Guo, Chunle and Han, Linghao and Jiang, Jun and Cheng, Ming-Ming and Gu, Jinwei and Loy, Chen Change},
  journal={arXiv:2104.10729},
  year={2021}
}

Contact Information

[email protected]

[email protected]
You might also like...
The all new way to turn your boring vector meshes into the new fad in town; Voxels!
The all new way to turn your boring vector meshes into the new fad in town; Voxels!

Voxelator The all new way to turn your boring vector meshes into the new fad in town; Voxels! Notes: I have not tested this on a rotated mesh. With fu

null 6 Feb 3, 2022
A dataset for online Arabic calligraphy
A dataset for online Arabic calligraphy

Calliar Calliar is a dataset for Arabic calligraphy. The dataset consists of 2500 json files that contain strokes manually annotated for Arabic callig

ARBML 114 Dec 28, 2022
A transformer which can randomly augment VOC format dataset (both image and bbox) online.
A transformer which can randomly augment VOC format dataset (both image and bbox) online.

VocAug It is difficult to find a script which can augment VOC-format dataset, especially the bbox. Or find a script needs complex requirements so it i

Coder.AN 1 Mar 5, 2022
Try out deep learning models online on Google Colab

Try out deep learning models online on Google Colab

Erdene-Ochir Tuguldur 1.5k Dec 27, 2022
Lyapunov-guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

PyTorch code to reproduce LyDROO algorithm [1], which is an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. It applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems and solves each subproblem via DROO algorithm. It includes:

Liang HUANG 87 Dec 28, 2022
Deep Reinforcement Learning for Multiplayer Online Battle Arena
Deep Reinforcement Learning for Multiplayer Online Battle Arena

MOBA_RL Deep Reinforcement Learning for Multiplayer Online Battle Arena Prerequisite Python 3 gym-derk Tensorflow 2.4.1 Dotaservice of TimZaman Seed R

Dohyeong Kim 32 Dec 18, 2022
Deep and online learning with spiking neural networks in Python
Deep and online learning with spiking neural networks in Python

Introduction The brain is the perfect place to look for inspiration to develop more efficient neural networks. One of the main differences with modern

Jason Eshraghian 447 Jan 3, 2023
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.
Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

This is the Vowpal Wabbit fast online learning code. Why Vowpal Wabbit? Vowpal Wabbit is a machine learning system which pushes the frontier of machin

Vowpal Wabbit 8.1k Jan 6, 2023
VIL-100: A New Dataset and A Baseline Model for Video Instance Lane Detection (ICCV 2021)

Preparation Please see dataset/README.md to get more details about our datasets-VIL100 Please see INSTALL.md to install environment and evaluation too

null 82 Dec 15, 2022
Comments
  • really nice platform

    really nice platform

    I think this platform makes people more convenient to implement using different kinds of method quickly, I want to ask a question of website building,Actually it is not about low light enhancement.

    I wonder how to build this user-friendly website (using API? or something, I am learning website building with HTTP CSS JS language) how to deal with those uploaded images? save to local drive or directly delete it? (what if there are lots of images uploaded at the same time, then how to keep this website working instead of crashing )

    opened by gravitychen 4
  • LOE metrics

    LOE metrics

    Hi! Thanks for the great survey!

    I found the LOE (Lightness Order Error) metric is not available now. It seems that I do not have the corresponding permission. Could you please provide it? Or is there any other way I can get it?

    opened by Arusa1 2
  • Downsampled output on LLIE-Platform

    Downsampled output on LLIE-Platform

    Hi, thank you for the great work of the LLIE-Platform! It is extremely helpful.

    However, I found the resolution of the output image has been downsampled for all methods. Is it possible to output the original resolution as the input?

    opened by Geniussh 1
  • Question about runtime measure

    Question about runtime measure

    Thanks for your amazing works.

    I'd like to know more about the code you used for measuring the running time.

    If it is not convenient for you to release the time testing code:

    1. What is the input for each model? Is it 1200*900*3 as in ZeroDCE++?
    2. How did you measure the training time for each model? Did you use torch.cuda.Event for time testing, or some other methods?
    3. Did you use any other ways to ensure the fairness of the comparison?

    I believe that posting these would be a great help for future works.

    Looking forward to your reply.

    opened by JizeZhangCS 0
Owner
Chongyi Li
Chongyi Li
GitHub
A new data augmentation method for extreme lighting conditions.

Random Shadows and Highlights This repo has the source code for the paper: Random Shadows and Highlights: A new data augmentation method for extreme l

Osama Mazhar 35 Nov 26, 2022
This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

Soubhik Sanyal 71 Dec 30, 2022
Deep Learning for 3D Point Clouds: A Survey (IEEE TPAMI, 2020)

??Deep Learning for 3D Point Clouds (IEEE TPAMI, 2020)

Qingyong 1.4k Jan 8, 2023
Official code for the CVPR 2022 (oral) paper "Extracting Triangular 3D Models, Materials, and Lighting From Images".

nvdiffrec Joint optimization of topology, materials and lighting from multi-view image observations as described in the paper Extracting Triangular 3D

NVIDIA Research Projects 1.4k Jan 1, 2023
A Lighting Pytorch Framework for Recommendation System, Easy-to-use and Easy-to-extend.

Torch-RecHub A Lighting Pytorch Framework for Recommendation Models, Easy-to-use and Easy-to-extend. 安装 pip install torch-rechub 主要特性 scikit-learn风格易用

Mincai Lai 67 Jan 4, 2023
Repository for the COLING 2020 paper "Explainable Automated Fact-Checking: A Survey."

Explainable Fact Checking: A Survey This repository and the accompanying webpage contain resources for the paper "Explainable Fact Checking: A Survey"

Neema Kotonya 42 Nov 17, 2022
This is the accompanying toolbox for the paper "A Survey on GANs for Anomaly Detection"

Anomaly detection using GANs.

Zuru Tech 77 Nov 30, 2022
Toontown: Galaxy, a new Toontown game based on Disney's Toontown Online

Toontown: Galaxy The official archive repo for Toontown: Galaxy, a new Toontown

null 1 Feb 15, 2022
Megaverse is a new 3D simulation platform for reinforcement learning and embodied AI research

Megaverse Megaverse is a new 3D simulation platform for reinforcement learning and embodied AI research. The efficient design of the engine enables ph

Aleksei Petrenko 191 Dec 23, 2022
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

H2O H2O is an in-memory platform for distributed, scalable machine learning. H2O uses familiar interfaces like R, Python, Scala, Java, JSON and the Fl

H2O.ai 6.1k Jan 5, 2023