A Survey on Deep Learning Technique for Video Segmentation

A Survey on Deep Learning Technique for Video Segmentation
Wenguan Wang, Tianfei Zhou, Fatih Porikli, David Crandall, and Luc Van Gool.

Contributing

Please feel free to create issues or pull requests to add papers.

Welcome any discussions on video segmentation at

1. Introduction

Video segmentation, i.e., partitioning video frames into multiple segments or objects, plays a critical role in a broad range of practical applications, from enhancing visual effects in movie, to understanding scenes in autonomous driving, to virtual background creation in video conferencing. In this survey, we comprehensively review two basic lines of research — video object segmentation and video semantic segmentation — by introducing their respective task settings, background concepts, perceived need, development history, and main challenges. In particular, we review eight sub-fields as given in the following figure:

2. Deep Learning-based Video Object Segmentation

3. Deep Learning-based Video Semantic Segmentation

4. Datasets

Popular Datasets in VOS and VSS

Citation

If you find our survey and repository useful for your research, please consider citing our paper:

@article{wang2021survey,
  title={A survey on deep learning technique for video segmentation},
  author={Wang, Wenguan and Zhou, Tianfei and Porikli, Fatih and Crandall, David and Van Gool, Luc},
  journal={arXiv preprint arXiv:2107.01153},
  year={2021}
}

An experimental technique for efficiently exploring neural architectures.

Great work! Could you please add papers from our group.

Hi! Dr.Tianfei. @tfzhou .

Could you consider adding three works from our groups on video detection/segmentation. Thanks!!

Video Object Detection by extending DETR. TransVOD: End-to-end Video Object Detection with Spatial-Temporal Transformers PAMI-2022, paper: https://arxiv.org/abs/2201.05047 code: https://github.com/SJTU-LuHe/TransVOD

Video Panopitc Segmentation using Query based approaches-CVPR-2022, paper: https://arxiv.org/abs/2204.04656 code: https://github.com/lxtGH/Video-K-Net

Video Instance Segmentation using Dynamic Network, PAMI-2022, paper: https://arxiv.org/abs/2107.13155 code: https://github.com/lxtGH/TemporalPyramidRouting

Thanks!!!

opened by lxtGH 1
Please consider adding RPCM (AAAI 2022) for semi-supervised video object segmentation, thanks!

Hi, thanks for the awesome survey & repo!

We believe a related semi-supervised video object segmentation work RPCM (AAAI2022, paper: https://arxiv.org/abs/2112.02853, code: https://github.com/JerryX1110/RPCMVOS) is missing, please consider add it, thanks!

opened by JerryX1110 1
Please consider add CrossVIS (ICCV 2021) for video instance segmentation

Hello, thanks for the awesome survey & repo!

We believe a related video instance segmentation work CrossVIS (ICCV 2021, paper: https://arxiv.org/abs/2104.05970, code: https://github.com/hustvl/CrossVIS) is missing, please consider add it, thanks!

opened by Yuxin-CV 1

A Survey on Deep Learning Technique for Video Segmentation

Related tags

Overview

A Survey on Deep Learning Technique for Video Segmentation

Contributing

1. Introduction

2. Deep Learning-based Video Object Segmentation

3. Deep Learning-based Video Semantic Segmentation

4. Datasets

Citation

You might also like...

An experimental technique for efficiently exploring neural architectures.

This project uses Template Matching technique for object detecting by detection of template image over base image.

PyExplainer: A Local Rule-Based Model-Agnostic Technique (Explainable AI)

This project uses Template Matching technique for object detecting by detection of template image over base image

PyExplainer: A Local Rule-Based Model-Agnostic Technique (Explainable AI)

[NeurIPS 2020] Blind Video Temporal Consistency via Deep Video Prior

[CVPR'21] Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild

Official implementation of the ICCV 2021 paper "Joint Inductive and Transductive Learning for Video Object Segmentation"

Dense Unsupervised Learning for Video Segmentation (NeurIPS*2021)

Comments

Great work! Could you please add papers from our group.

Please consider adding RPCM (AAAI 2022) for semi-supervised video object segmentation, thanks!

Please consider add CrossVIS (ICCV 2021) for video instance segmentation

Owner

Tianfei Zhou

Tensorflow Implementation of SMU: SMOOTH ACTIVATION FUNCTION FOR DEEP NETWORKS USING SMOOTHING MAXIMUM TECHNIQUE

Repository for the COLING 2020 paper "Explainable Automated Fact-Checking: A Survey."

This is the accompanying toolbox for the paper "A Survey on GANs for Anomaly Detection"

Video-Captioning - A machine Learning project to generate captions for video frames indicating the relationship between the objects in the video

An open source machine learning library for performing regression tasks using RVM technique.

Deep learning (neural network) based remote photoplethysmography: how to extract pulse signal from video using deep learning tools

We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.

CoReNet is a technique for joint multi-object 3D reconstruction from a single RGB image.

Pytorch implementation of AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks

Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equations