Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

Last update: Dec 15, 2022

Overview

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

This repository is an implementation of the method described in the following paper:

Shumian Xin, Neal Wadhwa, Tianfan Xue, Jonathan T. Barron, Pratul P. Srinivasan, Jiawen Chen, Ioannis Gkioulekas, and Rahul Garg. "Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image", ICCV 2021.

If you use our code or dataset, please cite our paper:

@article{Xin_2021_ICCV_dual_pixel,
    author    = {Xin, Shumian and Wadhwa, Neal and Xue, Tianfan and Barron, Jonathan T. and Srinivasan, Pratul P. and Chen, Jiawen and Gkioulekas, Ioannis and Garg, Rahul},
    title     = {Defocus Map Estimation and Deblurring From a Single Dual-Pixel Image},
    journal   = {IEEE/CVF International Conference on Computer Vision (ICCV)},
    year      = {2021}
}

Dataset

We captured a new dataset of 17 indoor and outdoor scenes using a Google Pixel 4 smartphone camera. Data can be found in ./DP_data_pixel_4. Google Pixel 4 camera provides dual-pixel (DP) images in the green channel. These DP images are 14-bit, with a black level of 1024. Please refer to this GitHub Repo for more details about Google Pixel's DP data.

We also provide calibrated blur kernels and vignetting patterns of our device in ./DP_data_pixel_4/calibration.

Code

Code implementation is in ./code. It is written in Python, with autograd package Jax. Note: When installing Jax, make sure to install with GPU support.

To reproduce results in the paper (in the paper, results are postprocessed for better visualization), run:

cd ./code; python ./run.py

Each optimization runs for 10,000 iterations with an Adam optimizer, and takes about 2 hours on an Nvidia Titan RTX GPU.

Code has been tested with:

Python 3.7.8
Jax 0.2.19
OpenCV 4.4.0

Note: To run the code on your own Google Pixel 4 data, please adjust the preprocessing step in ./code/util.py/load_data_and_calibration if needed, such that the input dual pixel images are normalized to the range of [0, 1].

You might also like...

Exploring Image Deblurring via Blur Kernel Space (CVPR'21)

Comments

Can you provide deblurred GT image in raw format?

Hello, thank you for the great work. It seems like the dataset in this repository is in raw format, different from the images in https://imaging.cs.cmu.edu/dual_pixels/supplement/001/index.html.

After running the code, how do we compare it with the groundtruth deblurred image (which is not in raw format). Do you apply gamma correction??

Thank you.

opened by jakeyoo87 2

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

Related tags

Overview

Defocus Map Estimation and Deblurring from a Single Dual-Pixel Image

Dataset

Code

You might also like...

Exploring Image Deblurring via Blur Kernel Space (CVPR'21)

Code for "PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation" CVPR 2019 oral

Deep Dual Consecutive Network for Human Pose Estimation (CVPR2021)

Drone-based Joint Density Map Estimation, Localization and Tracking with Space-Time Multi-Scale Attention Network

Code for Dual Contrastive Learning for Unsupervised Image-to-Image Translation, NTIRE, CVPRW 2021.

DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021)

Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes (CVPR2021)

Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

Honours project, on creating a depth estimation map from two stereo images of featureless regions

Comments

Can you provide deblurred GT image in raw format?

Owner

[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

Official respository for "Modeling Defocus-Disparity in Dual-Pixel Sensors", ICCP 2020

This is code to fit per-pixel environment map with spherical Gaussian lobes, using LBFGS optimization

Offical implementation for "Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation".

PyTorch code for 'Efficient Single Image Super-Resolution Using Dual Path Connections with Multiple Scale Learning'

SimDeblur is a simple framework for image and video deblurring, implemented by PyTorch

PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

A curated list of resources for Image and Video Deblurring

Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Image Deblurring using Generative Adversarial Networks