ABME (ICCV2021)
Junheum Park, Chul Lee, and Chang-Su Kim
Official PyTorch Code for "Asymmetric Bilateral Motion Estimation for Video Frame Interpolation" [paper]
Requirements
- PyTorch 1.7
- CUDA 11.0
- CuDNN 8.0.5
- python 3.8
Installation
Create conda environment:
$ conda create -n ABME python=3.8 anaconda
$ conda activate ABME
$ pip install opencv-python
$ conda install pytorch==1.7 torchvision cudatoolkit=11.0 -c pytorch
Download repository:
$ git clone https://github.com/JunHeum/ABME.git
Download pre-trained model parameters:
$ unzip ABME_Weights.zip
Check your nvcc
version:
$ nvcc --version
- To install correlation layer, you should match your
nvcc
version with cudatoolkit version of your conda environment. [nvcc_setting]
Install correlation layer:
$ cd correlation_package
$ python setup.py install
Quick Usage
Generate an intermediate frame on your pair of frames:
$ python run.py --first images/im1.png --second images/im3.png --output images/im2.png
Test
- Download the datasets.
- Copy the path of the test dataset. (e.g.,
/hdd/vimeo_interp_test
) - Parse this path into the
--dataset_root
argument. - (optional) You can ignore the
--is_save
. But, it yields a slightly different performance than evaluation on saved images.
$ python test.py --name ABME --is_save --Dataset ucf101 --dataset_root /where/is/your/ucf101_dataset/path
$ python test.py --name ABME --is_save --Dataset vimeo --dataset_root /where/is/your/vimeo_dataset/path
$ python test.py --name ABME --is_save --Dataset SNU-FILM-all --dataset_root /where/is/your/FILM_dataset/path
$ python test.py --name ABME --is_save --Dataset Xiph_HD --dataset_root /where/is/your/Xiph_dataset/path
$ python test.py --name ABME --is_save --Dataset X4K1000FPS --dataset_root /where/is/your/X4K1000FPS_dataset/path
Experimental Results
We provide interpolated frames on test datasets for fast comparison or users with limited GPU memory. Especially, the test on X4K1000FPS requires at least 20GB of GPU memory.
Train
We plan to share train codes soon!
Citation
Please cite the following paper if you feel this repository useful.
@inproceedings{park2021ABME,
author = {Park, Junheum and Lee, Chul and Kim, Chang-Su},
title = {Asymmetric Bilateral Motion Estimation for Video Frame Interpolation},
booktitle = {International Conference on Computer Vision},
year = {2021}
}
License
See MIT License