Revisiting Temporal Alignment for Video Restoration

Related tags

Deep Learning Revisiting-Temporal-Alignment-for-Video-Restoration
Overview

Revisiting Temporal Alignment for Video Restoration [arXiv]

Kun Zhou, Wenbo Li, Liying Lu, Xiaoguang Han, Jiangbo Lu

We provide our results at Google Cloud.

We will also to update this repository with the testing/training code later.

Citing

If you find this code useful for your research, please consider citing the following paper:

@article{zhou2021rta,
title={Revisiting Temporal Alignment for Video Restoration},
author={Kun Zhou and Wenbo Li and Liying Lu and Xiaoguang Han and Jiangbo Lu},
journal={arXiv preprint arXiv:2111.15288},
year={2021}
}

Comments
  • Iterative Alignment Module in Section 3.3

    Iterative Alignment Module in Section 3.3

    Thanks for your great job. It is kind for you to answer the following questions:

    1. Does IAM explore the optical flows produced by SpyNet or RAFT?
    2. Does IconVSR+IAM denote "replace optical flow for spatial alignment (including SpyNet) with IAM"?
    3. Does Iterative Number 3 in IAM mean "tmax=3", and is the total number of sub-alignments less than N(N+1)?
    4. Does DConv in Section 3.3.3 Eq 7,8 work the same as first-order flow-guided deformable alignment in BasicVSR++? Specifically, the estimated motion field h is served as the pre-offsets of deformable conv.
    opened by wdmwhh 5
  • Adaptive Re-weighting in Section 3.4 Eq 9-13

    Adaptive Re-weighting in Section 3.4 Eq 9-13

    Dear authors,

    I am very interested in your work and would appreciate if you find time to answer the following questions:

    1. As far as I understand, ARW block generates "refined aligned feature" for each frame feature k in {-N, N} set (see Eq 9-13). However, after such operator, how do you fuse all 2N+1 features into a single one?
    2. What is the "Reconstruction" module? What is the input for this module?
    3. In ablation studies, you have added "Baseline" model without IAM and ARW. However, it is not clear what operators have been utilized to fuse frame features? Do you simply concatenate features from frames without alignment?

    Thank you!

    opened by Magauiya 2
  • 您好，我想复现一篇论文代码但数据集链接已经失效，求助您谢谢！

    您好，我想复现一篇论文代码但数据集链接已经失效，求助您谢谢！

    您好我想复现这个代码 (https://github.com/cvlab-yonsei/dkn) 但原作者给出的数据集链接 (http://gofile.me/3G5St/2lFq5R3TL) 已经失效，但我看您在dkn项目的issue里说曾成功下载过作者使用的数据集，请问您还能找到当时下载的数据集嘛？可以的话您能否分享给我呢，非常感谢！！！

    opened by lzz7 2
  • Need your help with DKN(Deformable Kernel Network) dataset，please

    Need your help with DKN(Deformable Kernel Network) dataset，please

    Hello, nice to meet you. I try to run the code(https://github.com/cvlab-yonsei/dkn) , but the dataset gofile link (http://gofile.me/3G5St/2lFq5R3TL) is failed. Did you successfully downloaded the dataset? Would appreciate it if you can make a copy for me. I really need this. Thank you!!!

    opened by lzz7 0
  • Difficulties Running

    Difficulties Running

    I appreciate the code release, however, this was pretty difficult to get running. I managed to get there but the bar for the results was raised by the effort. I think it might be good to revisit a few things.

    Here's things that I think would help others be able to use it

    • Provide some information about your build environment/requirements
    • Make the DCNv2 a submodule and perhaps make the import work independent of location
    • Make symlinks in each model for libs and utils if you're going to import them in the scripts
    • Remove hard coded inputs, especially those pointing to your home directory etc
    opened by itsagoodbrain 5
Owner
null
GitHub
an implementation of Revisiting Adaptive Convolutions for Video Frame Interpolation using PyTorch

revisiting-sepconv This is a reference implementation of Revisiting Adaptive Convolutions for Video Frame Interpolation [1] using PyTorch. Given two f

Simon Niklaus 59 Dec 22, 2022
Revisiting Video Saliency: A Large-scale Benchmark and a New Model (CVPR18, PAMI19)

DHF1K =========================================================================== Wenguan Wang, J. Shen, M.-M Cheng and A. Borji, Revisiting Video Sal

Wenguan Wang 126 Dec 3, 2022
[NeurIPS 2020] Blind Video Temporal Consistency via Deep Video Prior

pytorch-deep-video-prior (DVP) Official PyTorch implementation for NeurIPS 2020 paper: Blind Video Temporal Consistency via Deep Video Prior TensorFlo

Yazhou XING 90 Oct 19, 2022
CVPR2021: Temporal Context Aggregation Network for Temporal Action Proposal Refinement

Temporal Context Aggregation Network - Pytorch This repo holds the pytorch-version codes of paper: "Temporal Context Aggregation Network for Temporal

Zhiwu Qing 63 Sep 27, 2022
Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment (ICCV2021)

Seeing Dynamic Scene in the Dark: High-Quality Video Dataset with Mechatronic Alignment This is a pytorch project for the paper Seeing Dynamic Scene i

DV Lab 21 Nov 28, 2022
BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment

Holy Wu 35 Jan 1, 2023
Official repository of "BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment"

BasicVSR_PlusPlus (CVPR 2022) [Paper] [Project Page] [Code] This is the official repository for BasicVSR++. Please feel free to raise issue related to

Kelvin C.K. Chan 227 Jan 1, 2023
Twins: Revisiting the Design of Spatial Attention in Vision Transformers

Twins: Revisiting the Design of Spatial Attention in Vision Transformers Very recently, a variety of vision transformer architectures for dense predic

null 482 Dec 18, 2022
Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations. [2021]

Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations This repo contains the Pytorch implementation of our paper: Revisit

Wouter Van Gansbeke 80 Nov 20, 2022
Official Code for ICML 2021 paper "Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline"

Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline Ankit Goyal, Hei Law, Bowei Liu, Alejandro Newell, Jia Deng Internati

Princeton Vision & Learning Lab 115 Jan 4, 2023
Revisiting, benchmarking, and refining Heterogeneous Graph Neural Networks.

Heterogeneous Graph Benchmark Revisiting, benchmarking, and refining Heterogeneous Graph Neural Networks. Roadmap We organize our repo by task, and on

THUDM 176 Dec 17, 2022
Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking

Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking We revisit and address issues with Oxford 5k and Paris 6k image retrieval benchm

Filip Radenovic 188 Dec 17, 2022
Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021)

Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme (NeurIPS2021) Overview Prerequisites Linux Pytho

Shaojie Li 34 Mar 31, 2022
RE-OWOD - Revisiting open world object detection

Revisting Open World Object Detection Installation See INSTALL.md. Dataset Our n

null 7 Jan 5, 2022
Multi-Stage Progressive Image Restoration

Multi-Stage Progressive Image Restoration Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Sh

Syed Waqas Zamir 859 Dec 22, 2022
Old Photo Restoration (Official PyTorch Implementation)

Bringing Old Photo Back to Life (CVPR 2020 oral)

Microsoft 11.3k Dec 30, 2022
(under submission) Bayesian Integration of a Generative Prior for Image Restoration

BIGPrior: Towards Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration Authors: Majed El Helou, and Sabine Süsstrunk {Note: p

Majed El Helou 22 Dec 17, 2022
Implementation for "Exploiting Aliasing for Manga Restoration" (CVPR 2021)

[CVPR Paper](To appear) | [Project Website](To appear) | BibTex Introduction As a popular entertainment art form, manga enriches the line drawings det

null 133 Dec 15, 2022
HINet: Half Instance Normalization Network for Image Restoration

HINet: Half Instance Normalization Network for Image Restoration Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, Chengpeng Chen Paper: https://arxiv.org

null 303 Dec 31, 2022