CVPR2021 Workshop - HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization.

Overview

HDRUNet [Paper Link]

HDRUNet: Single Image HDR Reconstruction with Denoising and Dequantization

By Xiangyu Chen, Yihao Liu, Zhengwen Zhang, Yu Qiao and Chao Dong

We won the second place in NTIRE2021 HDR Challenge (Track1: Single Frame). The paper is accepted to CVPR2021 Workshop.

BibTeX

@inproceedings{chen2021hdrunet,
  title={HDRUnet: Single image hdr reconstruction with denoising and dequantization},
  author={Chen, Xiangyu and Liu, Yihao and Zhang, Zhengwen and Qiao, Yu and Dong, Chao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={354--363},
  year={2021}
}

Overview

Overview of the network:

Overview of the loss function:

Tanh_L1(Y, H) = |Tanh(Y) - Tanh(H)|

Getting Started

  1. Dataset
  2. Configuration
  3. How to test
  4. How to train
  5. Visualization

Dataset

Register a codalab account and log in, then find the download link on this page:

https://competitions.codalab.org/competitions/28161#participate-get-data

It is strongly recommended to use the data provided by the competition organizer for training and testing, or you need at least a basic understanding of the competition data. Otherwise, you may not get the desired result.

Configuration

pip install -r requirements.txt

How to test

  • Modify dataroot_LQ and pretrain_model_G (you can also use the pretrained model which is provided in the ./pretrained_model) in ./codes/options/test/test_HDRUNet.yml, then run
cd codes
python test.py -opt options/test/test_HDRUNet.yml

The test results will be saved to ./results/testset_name.

How to train

  • Prepare the data. Modify input_folder and save_folder in ./scripts/extract_subimgs_single.py, then run
cd scripts
python extract_subimgs_single.py
  • Modify dataroot_LQ and dataroot_GT in ./codes/options/train/train_HDRUNet.yml, then run
cd codes
python train.py -opt options/train/train_HDRUNet.yml

The models and training states will be saved to ./experiments/name.

Visualization

In ./scripts, several scripts are available. data_io.py and metrics.py are provided by the competition organizer for reading/writing data and evaluation. Based on these codes, I provide a script for visualization by using the tone-mapping provided in metrics.py. Modify paths of the data in ./scripts/tonemapped_visualization.py and run

cd scripts
python tonemapped_visualization.py

to visualize the images.

Acknowledgment

The code is inspired by BasicSR.

Comments
  •  configuring dependencies & testing the setup

    configuring dependencies & testing the setup

    Hi, interesting work indeed! one basic question on config setup: requirement.txt mentions: numpy==1.18.5 torch==1.3.1+cu100 opencv_contrib_python==4.2.0.34

    However, above numpy version (1.18.5) is not compatible with python3.9 installed in my environment. For all above 3 requirements, do you recommend any specific python version (e.g. 3.6, 3.7 or 3.8 etc.)? would be good to know the version you had used..? Thank you!

    opened by ittim4 9
  • [About align ratio]

    [About align ratio]

    What is [align ratio] used for? I'm confused about the align-ratio... How to set the align ratio in train_HDRUNet.yml if train on custom dataset?

    #### datasets
    datasets:
      train:
        name: Single_LDR2HDR
        mode: LQGT_condition
        dataroot_LQ: /mnt/diskc/even/Ldr2HdrData/medium_sub
        dataroot_GT: /mnt/diskc/even/Ldr2HdrData/gt_sub
        dataroot_ratio: /data0/NTIRE2021_HDR/000_Train_SingleFrame_FirstStage/alignratio
        use_shuffle: true
        n_workers: 4 #  8
        batch_size: 16
        GT_size: 160
        use_flip: true
        use_rot: true
        condition: image
    
    opened by CaptainEven 6
  • How to get the final HDR image for visualization?

    How to get the final HDR image for visualization?

    Is the code

    img = io.imread_uint16_png(img_path, alignratio_path)
    img = np.clip(np.tanh(img), 0, 1) * 255
    img = img.round().astype(np.uint8)
    cv2.imwrite(save_path, cv2.cvtColor(img, cv2.COLOR_RGB2BGR))
    

    enough for converting output to HDR image?

    opened by CaptainEven 2
  • How to evaluate PSNR?

    How to evaluate PSNR?

    In your paper, do you directly use cropped Val dateset's PSNR as the PSNR in the final paper?
    In other areas, the best test results should be compared to real restored images.

    opened by jqtangust 1
  • Output question

    Output question

    Thank you for your great work! When i upload 1050 images to ./data,why there are only 11 processed images in ./results/HDRUNet_tanh_l1_bs_16_ps_160/000_Valid_SingleFrame_FirstStage?

    Another question: How can i modify the output path?

    opened by KaiDingyoo 1
  • Dataset

    Dataset

    Hello, thank you for your work, looks like you had a great competition with excellent results! I'm a student who is working on HDRI and want to ask if dataset of this model is still available anywhere because the competiton ended and I cannot reach dataset anymore. Thanks in advance!

    opened by orzanai 4
  • about alignratio in the training dataset

    about alignratio in the training dataset

    What does the value of alignratio in the training dataset mean? The range of gt_image/alignratio is not at [0,1]. What does the ratio do ? How are the alignratio in the train dataset derived?

    opened by sunyclj 1
  • get the HDR image in .hdr or .exr format

    get the HDR image in .hdr or .exr format

    Thanks for your impressive work!

    I have read your code and generate some results using the test.py file. It seems that the reconstructed hdr image is saved as uint16 png format, and in the tonemapped_visualization.py, the hdr image is tone mapped to LDR image for visualization. Am I right? And I also try to directly save the sr_img to .hdr or .exr format using the following code: cv2.imwrite('hdr_img_path.hdr', sr_img) The result seems good. Can I do so to get the HDR image in .hdr or .exr format instead of uint16 png format?

    Thank you~

    opened by Yliu724 4
  • GT data [understanding the max of range]

    GT data [understanding the max of range]

    First of all, thanks for your comments in this thread regarding align ratio. Wanted to understand few more details. For e.g.: How to find out what is the maximum nits represented in GT data? Is there any information or metadata indicated where GT is graded for say 1000nits, etc? Essentially, trying to understand whats the relationship of uint16 number in GT file and the nits it might represent. Current understanding is:: GT data is uint16 & gamma encoded and lets say if original range was [0 M], it is always scaled in a way that max of image content will be represented by 65535 where alignratio= 65535/M.

    Current thought process to make sense of GT data:

    1. GT_aligned = output of function imread_uint16_png() i.e. read GT uint16 from file and divide by alignratio to get original range, say [0 M].
    2. At this point, GT_aligned is not normalized to [0 1]. Ideally, would like to normalize it to [0 1] before applying gamma correction. Q1. How to normalize GT_aligned to [0 1]? e.g. GT_norm = function(GT_aligned)..what function to use for normalization?
    3. GT_linear = GT_norm ^ gamma
    4. Now, GT_linear will be in normalized to [0 1], but trying to understand what 1 represents? For e.g. for linear SDR signal, 1 will represent 100nits. Q2. But, for this GT_linear of HDR signal, 1 will translate to what value of nits? Any idea/ suggestions to understand this? or any assumptions made here?

    would be great if you can share your insight in this regard.

    opened by ittim4 6
  • Regarding training duration

    Regarding training duration

    Thank you so much for releasing this code!

    I had a question regarding the amount of time it takes to complete training. From the paper. I found the following information:

    All models are built on the PyTorch framework and trained with NVIDIA 2080Ti GPU. When the patch size of input is set to 256 x 256, the total training time is about 5 days.

    I have started training on my system with NVIDIA RTX 3090 GPU. However, after training for 6 days, I notice that only 410k iterations are completed and ~600k iterations are pending. I would expect training to be faster using the 3090 GPU card compared to the 2080Ti GPU card, but this seems to be extremely slow. I am using the code from this repository as is with no modifications apart from the training data paths.

    Could you please let me know if I am missing something? Is some additional step required in order to accelerate the training?

    Thank you.

    opened by Kadakol 2
Owner
XyChen
PhD. Student,Computer Vision
XyChen
Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set (CVPRW 2019). A PyTorch implementation.

Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set —— PyTorch implementation This is an unofficial offici

Sicheng Xu 833 Dec 28, 2022
Contrastive Learning for Compact Single Image Dehazing, CVPR2021

AECR-Net Contrastive Learning for Compact Single Image Dehazing, CVPR2021. Official Pytorch based implementation. Paper arxiv Pytorch Version TODO: mo

glassy 253 Jan 1, 2023
Model-based 3D Hand Reconstruction via Self-Supervised Learning, CVPR2021

S2HAND: Model-based 3D Hand Reconstruction via Self-Supervised Learning S2HAND presents a self-supervised 3D hand reconstruction network that can join

Yujin Chen 72 Dec 12, 2022
A Planar RGB-D SLAM which utilizes Manhattan World structure to provide optimal camera pose trajectory while also providing a sparse reconstruction containing points, lines and planes, and a dense surfel-based reconstruction.

ManhattanSLAM Authors: Raza Yunus, Yanyan Li and Federico Tombari ManhattanSLAM is a real-time SLAM library for RGB-D cameras that computes the camera

null 117 Dec 28, 2022
PyTorch implementation of Deep HDR Imaging via A Non-Local Network (TIP 2020).

NHDRRNet-PyTorch This is the PyTorch implementation of Deep HDR Imaging via A Non-Local Network (TIP 2020). 0. Differences between Original Paper and

Yutong Zhang 1 Mar 1, 2022
Pytorch implementation of various High Dynamic Range (HDR) Imaging algorithms

Deep High Dynamic Range Imaging Benchmark This repository is the pytorch impleme

Tianhong Dai 5 Nov 16, 2022
CoReNet is a technique for joint multi-object 3D reconstruction from a single RGB image.

CoReNet CoReNet is a technique for joint multi-object 3D reconstruction from a single RGB image. It produces coherent reconstructions, where all objec

Google Research 80 Dec 25, 2022
Official repository for "Restormer: Efficient Transformer for High-Resolution Image Restoration". SOTA for motion deblurring, image deraining, denoising (Gaussian/real data), and defocus deblurring.

Restormer: Efficient Transformer for High-Resolution Image Restoration Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan,

Syed Waqas Zamir 906 Dec 30, 2022
"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

Yuanhao Cai 274 Jan 5, 2023
MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera

MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera

Felix Wimbauer 494 Jan 6, 2023
Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images (ICCV 2021)

Table of Content Introduction Getting Started Datasets Installation Experiments Training & Testing Pretrained models Texture fine-tuning Demo Toward R

VinAI Research 42 Dec 5, 2022
Code for "Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency" paper

UNICORN ?? Webpage | Paper | BibTex PyTorch implementation of "Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency" pap

null 118 Jan 6, 2023
OcclusionFusion: realtime dynamic 3D reconstruction based on single-view RGB-D

OcclusionFusion (CVPR'2022) Project Page | Paper | Video Overview This repository contains the code for the CVPR 2022 paper OcclusionFusion, where we

Wenbin Lin 193 Dec 15, 2022
Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of images as "pixels"

picinpics Script that receives an Image (original) and a set of images to be used as "pixels" in reconstruction of the Original image using the set of

RodrigoCMoraes 1 Oct 24, 2021
Official implementation of Deep Convolutional Dictionary Learning for Image Denoising.

DCDicL for Image Denoising Hongyi Zheng*, Hongwei Yong*, Lei Zhang, "Deep Convolutional Dictionary Learning for Image Denoising," in CVPR 2021. (* Equ

Z80 91 Dec 21, 2022
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

Kai Zhang 1.2k Dec 29, 2022
This is an official implementation of the CVPR2022 paper "Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots".

Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spots Blind2Unblind Citing Blind2Unblind @inproceedings{wang2022blind2unblind, tit

demonsjin 58 Dec 6, 2022