Official repository for "Deep Recurrent Neural Network with Multi-scale Bi-directional Propagation for Video Deblurring".

Related tags

Deep Learning RNN-MBP
Overview

RNN-MBP

Deep Recurrent Neural Network with Multi-scale Bi-directional Propagation for Video Deblurring (AAAI-2022)
by Chao Zhu, Hang Dong, Jinshan Pan, Boyang Liang, Yuhao Huang, Lean Fu, and Fei Wang

[Paper] [Supp]

Results

Results on GOPRO

image image

Results on DVD

image

Results on RBVD

image

Prerequisites

  • Python 3.6
  • PyTorch 1.8
  • opencv-python
  • scikit-image
  • lmdb
  • thop
  • tqdm
  • tensorboard

Real-world Bluryy Video Dataset (RBVD)

We have collected a new RBVD dataset with more scenes and perfect alignment, using the proposed Digital Video Acquisition System.

Training

Please download and unzip the dataset file for each benchmark.

Then, specify the <path> (para.data_root) where you put the dataset file and the corresponding dataset configurations in the command (e.g. para.dataset=gopro or gopro_ds_lmdb).

The default training process requires at least 4 NVIDIA Tesla V100 32Gb GPUs.

The training command is shown below:

python main.py --data_root <path> --dataset gopro_ds_lmdb  --num_gpus 4 --batch_size 4  --patch_size [256, 256]  --end_epoch 500

Testing

Please download checkpoints and unzip it under the Source directory.

Example command to run a pre-trained model:

python test.py --data_root <path> --dataset gopro_ds_lmdb  --test_only --test_checkpoint <path>  --model RNN-MBP

Citing

If you use any part of our code, or RNN-MBP and RBVD are useful for your research, please consider citing:

@inproceedings{chao2022,
  title={Deep Recurrent Neural Network with Multi-scale Bi-directional Propagation for Video Deblurring},
  author={Chao, Zhu and Hang, Dong and Jinshan, Pan and Boyang, Liang and Yuhao, Huang and Lean, Fu and Fei, Wang},
  booktitle={AAAI},
  year={2022},
}
You might also like...
The official repository for BaMBNet

BaMBNet-Pytorch Paper

Junjun Jiang 18 Dec 4, 2022
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

selfcontact This repo is part of our project: On Self-Contact and Human Pose. [Project Page] [Paper] [MPI Project Page] It includes the main function

Lea Müller 68 Dec 6, 2022
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.
CVPR 2021 - Official code repository for the paper: On Self-Contact and Human Pose.

SMPLify-XMC This repo is part of our project: On Self-Contact and Human Pose. [Project Page] [Paper] [MPI Project Page] License Software Copyright Lic

Lea Müller 83 Dec 14, 2022
This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.
This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard evaluation metric to measure the accuracy and robustness of 3D face reconstruction methods from a single image under variations in viewing angle, lighting, and common occlusions.

NoW Evaluation This is the official repository for evaluation on the NoW Benchmark Dataset. The goal of the NoW benchmark is to introduce a standard e

Soubhik Sanyal 71 Dec 30, 2022
Official repository for the paper "Going Beyond Linear Transformers with Recurrent Fast Weight Programmers"

Recurrent Fast Weight Programmers This is the official repository containing the code we used to produce the experimental results reported in the pape

IDSIA 36 Nov 15, 2022
Official repository for the paper "Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks"

Easy-To-Hard The official repository for the paper "Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks". Gett

Avi Schwarzschild 52 Sep 8, 2022
Official repository for the CVPR 2021 paper "Learning Feature Aggregation for Deep 3D Morphable Models"

Deep3DMM Official repository for the CVPR 2021 paper Learning Feature Aggregation for Deep 3D Morphable Models. Requirements This code is tested on Py

null 38 Dec 27, 2022
Official repository for the paper, MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Understanding.
Official repository for the paper, MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Understanding.

MidiBERT-Piano Authors: Yi-Hui (Sophia) Chou, I-Chun (Bronwin) Chen Introduction This is the official repository for the paper, MidiBERT-Piano: Large-

null 137 Dec 15, 2022
Official repository of my book:
Official repository of my book: "Deep Learning with PyTorch Step-by-Step: A Beginner's Guide"

This is the official repository of my book "Deep Learning with PyTorch Step-by-Step". Here you will find one Jupyter notebook for every chapter in the book.

Daniel Voigt Godoy 340 Jan 1, 2023
Comments
  • Problems for testing

    Problems for testing

    Hello,

    I test your model with the released checkpoints, I rename the checkpoints as 'model_best.pth.tar' and run by the following command:

    'python test.py --data_root gopro_lmdb --dataset gopro_ds_lmdb --test_only --test_checkpoint ./model_best.pth.tar --model RNN-MBP'

    However, I can not load the checkpoints successfully, some errors (I list parts of them) come up as:

    Missing key(s) in state_dict: "module.model.forward_encoder.skip_enc1.weight", "module.model.forward_encoder.skip_enc2.weight", "module.model.forward_encoder.skip_enc3.weight", "module.model.forward_encoder.skip_dec1.weight", "module.model.forward_encoder.skip_dec2.weight", "module.model.forward_encoder.skip_dec3.weight"

    Unexpected key(s) in state_dict: "module.model.ORSNet.orb1.body.0.CA.conv_du.0.weight", "module.model.ORSNet.orb1.body.0.CA.conv_du.2.weight", "module.model.ORSNet.orb1.body.0.body.0.weight", "module.model.ORSNet.orb1.body.0.body.1.weight", "

    How can I fix it? Thank you very much!

    opened by pp00704831 1
  • Slow performance?

    Slow performance?

    Thanks for sharing this fantastic work. I'm wondering if it's typical for a 1280x720 image to take 30 seconds to process. Could it be that my GPU is not being utilized properly? When I run the same images with ESTRNN it only takes 1 or 2 seconds per frame.

    "2022-01-15 23:44:20.211283: W c:\tf_jenkins\home\workspace\release-win\m\windows\py\36\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations. 2022-01-15 23:44:20.211577: W c:\tf_jenkins\home\workspace\release-win\m\windows\py\36\tensorflow\core\platform\cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations. [*] Reading checkpoints... [*] Reading intermediate checkpoints... Success Saving results: ./testing_res\00000000.png ... 29.047s Saving results: ./testing_res\00000001.png ... 28.678s"

    opened by noobtoob4lyfe 1
Owner
SIV-LAB
SIV-LAB
GitHub
Official repository for Jia, Raghunathan, Göksel, and Liang, "Certified Robustness to Adversarial Word Substitutions" (EMNLP 2019)

Certified Robustness to Adversarial Word Substitutions This is the official GitHub repository for the following paper: Certified Robustness to Adversa

Robin Jia 38 Oct 16, 2022
Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology

Official repository for the ICLR 2021 paper Evaluating the Disentanglement of Deep Generative Models with Manifold Topology Sharon Zhou, Eric Zelikman

Stanford Machine Learning Group 34 Nov 16, 2022
The repository offers the official implementation of our paper in PyTorch.

Cloth Interactive Transformer (CIT) Cloth Interactive Transformer for Virtual Try-On Bin Ren1, Hao Tang1, Fanyang Meng2, Runwei Ding3, Ling Shao4, Phi

Bingoren 49 Dec 1, 2022
Official code repository of the paper Learning Associative Inference Using Fast Weight Memory by Schlag et al.

Learning Associative Inference Using Fast Weight Memory This repository contains the offical code for the paper Learning Associative Inference Using F

Imanol Schlag 18 Oct 12, 2022
Official repository for "Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems"

Action-Based Conversations Dataset (ABCD) This respository contains the code and data for ABCD (Chen et al., 2021) Introduction Whereas existing goal-

ASAPP Research 49 Oct 9, 2022
Official repository for HOTR: End-to-End Human-Object Interaction Detection with Transformers (CVPR'21, Oral Presentation)

Official PyTorch Implementation for HOTR: End-to-End Human-Object Interaction Detection with Transformers (CVPR'2021, Oral Presentation) HOTR: End-to-

Kakao Brain 114 Nov 28, 2022
Official repository for "Intriguing Properties of Vision Transformers" (2021)

Intriguing Properties of Vision Transformers Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, & Ming-Hsuan Yang P

Muzammal Naseer 155 Dec 27, 2022
Competitive Programming Club, Clinify's Official repository for CP problems hosting by club members.

Clinify-CPC_Programs This repository holds the record of the competitive programming club where the competitive coding aspirants are thriving hard and

Clinify Open Sauce 4 Aug 22, 2022
Official repository for "On Improving Adversarial Transferability of Vision Transformers" (2021)

Improving-Adversarial-Transferability-of-Vision-Transformers Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Fahad Khan, Fatih Porikli arxiv link A

Muzammal Naseer 47 Dec 2, 2022
This is the official repository of XVFI (eXtreme Video Frame Interpolation)

XVFI This is the official repository of XVFI (eXtreme Video Frame Interpolation), https://arxiv.org/abs/2103.16206 Last Update: 20210607 We provide th

Jihyong Oh 195 Dec 29, 2022