SimDeblur is a simple framework for image and video deblurring, implemented by PyTorch

Overview

SimDeblur

SimDeblur (Simple Deblurring) is an open source framework for image and video deblurring toolbox based on PyTorch, which contains most deep-learning based state-of-the-art deblurring algorithms. It is easy to implement your own image or video deblurring or other restoration algorithms.

Major features

  • Modular Design

The toolbox decomposes the deblurring framework into different components and one can easily construct a customized restoration framework by combining different modules.

  • State of the art

The toolbox contains most deep-learning based state-of-the-art deblurring algorithms, including MSCNN, SRN, DeblurGAN, EDVR, etc.

  • Distributed Training

New Features

[2021/3/31] support DVD, GoPro and REDS video deblurring datasets. [2021/3/21] first release.

Surpported Methods and Benchmarks

Dependencies and Installation

  • Python 3 (Conda is recommended)
  • Pytorch 1.5.1 (with GPU)
  • CUDA 10.2+
  1. Clone the repositry or download the zip file
     git clone https://github.com/ljzycmd/SimDeblur.git
    
  2. Install SimDeblur
    # create a pytorch env
    conda create -n simdeblur python=3.7
    conda activate simdeblur   
    # install the packages
    cd SimDeblur
    bash Install.sh

Usage

1 Start with trainer

You can construct a simple training process use the default trainer like following:

from simdeblur.config import build_config, merge_args
from simdeblur.engine.parse_arguments import parse_arguments
from simdeblur.engine.trainer import Trainer


args = parse_arguments()

cfg = build_config(args.config_file)
cfg = merge_args(cfg, args)
cfg.args = args

trainer = Trainer(cfg)
trainer.train()

Then start training with single GPU:

CUDA_VISIBLE_DEVICES=0 bash ./tools/train.sh ./config/dbn/dbn_dvd.yaml 1

multi GPU training:

CUDA_VISIBLE_DEVICES=0,1,2,3 bash ./tools/train.sh ./config/dbn/dbn_dvd.yaml 4

2 Build each module

The SimDeblur also provides you to build each module. build the a dataset:

from easydict import EasyDict as edict
from simdeblur.dataset import build_dataset

dataset = build_dataset(edict({
    "name": "DVD",
    "mode": "train",
    "sampling": "n_c",
    "overlapping": True,
    "interval": 1,
    "root_gt": "./dataset/DVD/quantitative_datasets",
    "num_frames": 5,
    "augmentation": {
        "RandomCrop": {
            "size": [256, 256] },
        "RandomHorizontalFlip": {
            "p": 0.5 },
        "RandomVerticalFlip": {
            "p": 0.5 },
        "RandomRotation90": {
            "p": 0.5 },
    }
}))

print(dataset[0])

build the model:

from simdeblur.model import build_backbone

model = build_backbone({
    "name": "DBN",
    "num_frames": 5,
    "in_channels": 3,
    "inner_channels": 64
})

x = torch.randn(1, 5, 3, 256, 256)
out = model(x)

build the loss:

from simdeblur.model import build_loss

criterion = build_loss({
    "name": "MSELoss",
})
x = torch.randn(2, 3, 256, 256)
y = torch.randn(2, 3, 256, 256)
print(criterion(x, y))

And the optimizer and lr_scheduler also can be created by "build_optimizer" and "build_lr_scheduler" etc.

Dataset Description

Click here for more information.

Acknowledgment

[1] facebookresearch. detectron2. https://github.com/facebookresearch/detectron2

[2] subeeshvasu. Awesome-Deblurring. https://github.com/subeeshvasu/Awesome-Deblurring

Citations

If SimDeblur helps your research or work, please consider citing SimDeblur.

@misc{cao2021simdeblur,
  author =       {Mingdeng Cao},
  title =        {SimDeblur},
  howpublished = {\url{https://github.com/ljzycmd/SimDeblur}},
  year =         {2021}
}

If you have any question, please contact me at mingdengcao AT gmail.com.

Comments
  • No object names xxx found in backbone registry!

    No object names xxx found in backbone registry!

    I cannot run mscnn and srn model with the error:

    Traceback (most recent call last):
      File "train.py", line 28, in <module>
        main()
      File "train.py", line 23, in main
        trainer = Trainer(cfg)
      File "/deblur/SimDeblur/simdeblur/engine/trainer.py", line 69, in __init__
        self.model = self.build_model(cfg).to(self.device)
      File "/deblur/SimDeblur/simdeblur/engine/trainer.py", line 305, in build_model
        model = build_backbone(cfg.model)
      File "/SimDeblur/simdeblur/model/build.py", line 26, in build_backbone
        return build(cfg, BACKBONE_REGISTRY)
      File "/SimDeblur/simdeblur/model/build.py", line 21, in build
        ret = registry.get(name)(**args)
      File "/SimDeblur/utils/registry.py", line 34, in get
        raise KeyError("No object names {} found in {} registry!".format(name, self._name))
    KeyError: 'No object names MSResNet found in backbone registry!'
    
    opened by INVOKERer 3
  • Hi. I'm so sorry to bother you,i hope you can share me a Pretrained model link.but,in your code of NonUniformBlurKernelEstimation ,your Pretrained model link is not valid, can you provide a new link?

    Hi. I'm so sorry to bother you,i hope you can share me a Pretrained model link.but,in your code of NonUniformBlurKernelEstimation ,your Pretrained model link is not valid, can you provide a new link?

    Hi. I'm so sorry to bother you,i see you stared a github project of NonUniformBlurKernelEstimation in paper of Non-uniform Blur Kernel Estimation via Adaptive Basis Decomposition,but now the Pretrained model link is not valid and can't connected author,so can you provide a new link?

    opened by panyangh 1
  • Integrating blur decomposition works and real-world benchmarks

    Integrating blur decomposition works and real-world benchmarks

    Hi, great work!

    Any interests to integrate blur decomposition works into this general framework, such as: Learning to Extract a Video Sequence from a Single Motion-Blurred Image (CVPR 2018) (Image) Bringing Alive Blurred Moments (CVPR 2019) (Image) Learning to Extract Flawless Slow Motion From Blurry Videos (CVPR 2019) (Video) Blurry Video Frame Interpolation (CVPR 2020) (Video)

    Also, real-world benchmarks such as: RealBlur (Image) BSD (Video)

    If you would like to do it, maybe I can offer some help :)

    Email: [email protected] WeChat: zzh-tech

    opened by zzh-tech 0
  • Using the other supported neural network models on ColabNotebook and inference_image.py

    Using the other supported neural network models on ColabNotebook and inference_image.py

    Hello there,

    I was able to follow your example which you have posted on the ColabNotebook and have successfully able to perform deblurring using the DBN model on the test images locally on my PC via JupyterNotebook with CUDA enabled on PyTorch. So the example in ColabNotebook using DBN is working well.

    Next, I tried to load a different mode (i.e. the DBLRNet) to compare the results, with the code snippet below.

    ...
    model = build_backbone(model_cfg)
    ckpt = torch.load("./demo/dblrnet_dvd.pth")
    model_ckpt = ckpt["model"]
    model_ckpt = {k[7:]: v for k, v in model_ckpt.items()}
    model.load_state_dict(model_ckpt)
    model = model.to(device)
    ...
    

    I then get an error from the Python below.

    ---------------------------------------------------------------------------
    RuntimeError                              Traceback (most recent call last)
    /tmp/ipykernel_3448/3417428220.py in <module>
         19 model_ckpt = ckpt["model"]
         20 model_ckpt = {k[7:]: v for k, v in model_ckpt.items()}
    ---> 21 model.load_state_dict(model_ckpt)
         22 model = model.to(device)
    
    ~/anaconda3/envs/simdeblur/lib/python3.7/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
       1666         if len(error_msgs) > 0:
       1667             raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
    -> 1668                                self.__class__.__name__, "\n\t".join(error_msgs)))
       1669         return _IncompatibleKeys(missing_keys, unexpected_keys)
       1670 
    
    RuntimeError: Error(s) in loading state_dict for DBN:
    	Missing key(s) in state_dict: "F0.0.weight", "F0.0.bias", "F0.1.weight", "F0.1.bias", "F0.1.running_mean", "F0.1.running_var", "D1.0.weight", "D1.0.bias", "D1.1.weight", "D1.1.bias", "D1.1.running_mean", "D1.1.running_var", "F1_1.0.weight", "F1_1.0.bias", "F1_1.1.weight", "F1_1.1.bias", "F1_1.1.running_mean", "F1_1.1.running_var",
    ...
    

    I have also attempted to run the inference_image.py script to do the same thing with following command in Linux. python inference_image.py ./configs/dblrnet/dblrnet_dvd.yaml ./demo/dblrnet_dvd.pth --img=./datasets/input/00000.jpg

    This resulted in the following error below.

    Using checkpoint loaded from ./demo/dblrnet_dvd.pth for testing.
    Traceback (most recent call last):
      File "inference_image.py", line 81, in <module>
        inference()
      File "inference_image.py", line 70, in inference
        outputs = arch.postprocess(arch.model(arch.preprocess(input_image)))
      File "/home/emui/anaconda3/envs/simdeblur/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/emui/sandbox-git/SimDeblur/simdeblur/model/backbone/dblrnet/dblrnet.py", line 52, in forward
        l2 = self.L_in(x)
      File "/home/emui/anaconda3/envs/simdeblur/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/emui/anaconda3/envs/simdeblur/lib/python3.7/site-packages/torch/nn/modules/container.py", line 204, in forward
        input = module(input)
      File "/home/emui/anaconda3/envs/simdeblur/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
        return forward_call(*input, **kwargs)
      File "/home/emui/anaconda3/envs/simdeblur/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 613, in forward
        return self._conv_forward(input, self.weight, self.bias)
      File "/home/emui/anaconda3/envs/simdeblur/lib/python3.7/site-packages/torch/nn/modules/conv.py", line 609, in _conv_forward
        input, weight, bias, self.stride, self.padding, self.dilation, self.groups
    RuntimeError: Calculated padded input size per channel: (1 x 722 x 1282). Kernel size: (3 x 3 x 3). Kernel size can't be greater than actual input size
    

    Do you know what could be causing the issue here? It would be really nice if you could provide step-by-step examples on how to use the scripts to deblur images/videos along with test inputs and expected output, so that we know we have properly setup the SimDeblur on our machines locally.

    P.S. It would be useful to also add in the README.md on the instructions of how to install all the required dependencies for SimDeblur, e.g. the Python packages and the CUDA AI libraries and toolkit on Linux. Thanks again for the great work. :+1:

    opened by dariuskrail 4
  • Have you trained the model with your own script?

    Have you trained the model with your own script?

    Hello, thanks for your contributions. Have you trained the model with your own script, or you just copied or wrote the code for that method without training by yourself? If not, how to verify the correctness of your tool? Thanks.

    opened by c-yn 1
  • Unable to execute inference_image script

    Unable to execute inference_image script

    Hi, Great repository, just loved it. I am trying to execute inference_image.py with the dbn architecture as the backbone but I'm getting stuck on this: Executing the script produces the following:

    Cannot inport EDVR modules!!! Cannot import STFAN modules!!! Using checkpoint loaded from ./checkpoints/dbn_ckpt.pth for testing. Traceback (most recent call last): File "inference_image.py", line 81, in inference() File "inference_image.py", line 70, in inference outputs = arch.postprocess(arch.model(arch.preprocess(input_image))) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1102, in _call_impl return forward_call(*input, **kwargs) File "/content/drive/MyDrive/SimDeblur/simdeblur/model/backbone/dbn/dbn.py", line 137, in forward central_frame = x[:, self.num_frames // 2] IndexError: index 2 is out of bounds for dimension 1 with size 1

    I tried printing the dimensions of the image and it came out to be: torch.Size([1, 1, 3, 385, 1504]) and self.num_frames is 5 for my case. Don't know how to resolve the issue. Please help out.

    opened by Aparajit-Garg 1
  • Checkpoints for ESTRNN

    Checkpoints for ESTRNN

    Are there plans to add pre-trained model for ESTRNN? The ones provided on the official ESTRNN github produce artifacts in my tests so far. Would be interested to know if there are plans to add models trained of REDS or GoPro.

    opened by noobtoob4lyfe 1
  • Colab Questions

    Colab Questions

    Thank you so much for sharing the this! Forgive me, because I'm not super technical but I was hoping you could walk me through how to use the Colab notebook to test my own image sequence. When I "run all" in Colab it shows a single output comparison jpeg. Does it save the entire deblurred image sequence anywhere? All I can see are the input images when I search the folder structure. How would I deblur and entire image sequence and save the resulting images? Thanks in advance!

    opened by noobtoob4lyfe 4
Owner
null
A curated list of resources for Image and Video Deblurring

A curated list of resources for Image and Video Deblurring

Subeesh Vasu 1.7k Jan 1, 2023
PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

null 11 May 19, 2022
Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

This repository is the official PyTorch implementation of Cascaded Deep Video Deblurring Using Temporal Sharpness Prior and Non-local Spatial-Temporal Similarity

hippopmonkey 4 Dec 11, 2022
[CVPR 2021] Official PyTorch Implementation for "Iterative Filter Adaptive Network for Single Image Defocus Deblurring"

IFAN: Iterative Filter Adaptive Network for Single Image Defocus Deblurring Checkout for the demo (GUI/Google Colab)! The GUI version might occasional

Junyong Lee 173 Dec 30, 2022
Image Deblurring using Generative Adversarial Networks

DeblurGAN arXiv Paper Version Pytorch implementation of the paper DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks. Our netwo

Orest Kupyn 2.2k Jan 1, 2023
Exploring Image Deblurring via Blur Kernel Space (CVPR'21)

Exploring Image Deblurring via Encoded Blur Kernel Space About the project We introduce a method to encode the blur operators of an arbitrary dataset

VinAI Research 118 Dec 19, 2022
[ICCV 2021] Official Tensorflow Implementation for "Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions"

KPAC: Kernel-Sharing Parallel Atrous Convolutional block This repository contains the official Tensorflow implementation of the following paper: Singl

Hyeongseok Son 50 Dec 29, 2022
DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021)

Evaluation, Training, Demo, and Inference of DeFMO DeFMO: Deblurring and Shape Recovery of Fast Moving Objects (CVPR 2021) Denys Rozumnyi, Martin R. O

Denys Rozumnyi 139 Dec 26, 2022
Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes (CVPR2021)

RSCD (BS-RSCD & JCD) Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes (CVPR2021) by Zhihang Zhong, Yinqiang Zheng, Imari Sato We co

null 81 Dec 15, 2022
In this project we investigate the performance of the SetCon model on realistic video footage. Therefore, we implemented the model in PyTorch and tested the model on two example videos.

Contrastive Learning of Object Representations Supervisor: Prof. Dr. Gemma Roig Institutions: Goethe University CVAI - Computational Vision & Artifici

Dirk Neuhäuser 6 Dec 8, 2022
Reinforcement learning framework and algorithms implemented in PyTorch.

Reinforcement learning framework and algorithms implemented in PyTorch.

Robotic AI & Learning Lab Berkeley 2.1k Jan 4, 2023
Video-Captioning - A machine Learning project to generate captions for video frames indicating the relationship between the objects in the video

Video-Captioning - A machine Learning project to generate captions for video frames indicating the relationship between the objects in the video

null 1 Jan 23, 2022
ChatBot-Pytorch - A GPT-2 ChatBot implemented using Pytorch and Huggingface-transformers

ChatBot-Pytorch A GPT-2 ChatBot implemented using Pytorch and Huggingface-transf

ParZival 42 Dec 9, 2022
An end-to-end PyTorch framework for image and video classification

What's New: March 2021: Added RegNetZ models November 2020: Vision Transformers now available, with training recipes! 2020-11-20: Classy Vision v0.5 R

Facebook Research 1.5k Dec 31, 2022
We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.

Multi-Modal Self-Supervision using GDT and StiCa This is an official pytorch implementation of papers: Multi-modal Self-Supervision from Generalized D

Facebook Research 42 Dec 9, 2022
Simple-Image-Classification - Simple Image Classification Code (PyTorch)

Simple-Image-Classification Simple Image Classification Code (PyTorch) Yechan Kim This repository contains: Python3 / Pytorch code for multi-class ima

Yechan Kim 8 Oct 29, 2022
Many Class Activation Map methods implemented in Pytorch for CNNs and Vision Transformers. Including Grad-CAM, Grad-CAM++, Score-CAM, Ablation-CAM and XGrad-CAM

Class Activation Map methods implemented in Pytorch pip install grad-cam ⭐ Tested on many Common CNN Networks and Vision Transformers. ⭐ Includes smoo

Jacob Gildenblat 6.6k Jan 6, 2023
Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021) This repository is the official P

Jingyun Liang 159 Dec 30, 2022