implement of SwiftNet:Real-time Video Object Segmentation

haochen wang

Last update: Dec 14, 2022

Related tags

Deep Learning SwiftNet

Overview

SwiftNet

The official PyTorch implementation of SwiftNet:Real-time Video Object Segmentation, which has been accepted by CVPR2021.

Requirements

Python >= 3.6
Pytorch 1.5
Numpy
Pillow
opencv-python
scipy
tqdm

Training

The training pipeline of Swiftnet is similar with the training pipeline of STM, which can be found in our reproduced STM training code.

Inference

Usage

python eval.py -g 0 -y 17 -s val -D 'path to davis'

Performance

Performance on Davis-17 val set.

backbone	J&F	J	F	FPS	weights
resnet-18	77.6	75.5	79.7	65	`link`

Note: The FPS is tested on one P100, which does not include the time of image loading and evaluation cost.

Acknowledgement

This repository is partially founded on the official STM repository.

Citation

If you find this repository helpful and want to cite SwiftNet in your own projects, please use the following citation info.

@inproceedings{wang2021swiftnet,
  title={SwiftNet: Real-time Video Object Segmentation},
  author={Wang, Haochen and Jiang, Xiaolong and Ren, Haibing and Hu, Yao and Bai, Song},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={1296--1305},
  year={2021}
}

Comments

Resnet18 weights

Hi,thank you for your paper! Your work is very nice. I have a question,your link which refers to 'swiftnet_resnet18_old.pth' is none,could you please add this?

opened by jiangxiaobaiup 1
Variation-Aware Trigger?

Hi, thank you for the code & paper. I am looking through the code and cannot seem to find the implementation of the VAT. It appears to me that the memory scheduling is fixed: https://github.com/haochenheheda/SwiftNet/blob/02c7f449787f4c1c1ad4d21136c3870049725848/eval.py#L42

Am I missing something? Also, does y_t in the paper refer to the binarized "hard" mask or soft mask? Thanks!

opened by hkchengrex 0
training pipeline

Hi, first thanks for your great work! Is it possible to also upload your training pipeline? That would be very helpful, because I'm troubling with creating a correct working one. thanks!

opened by grewanhassan 0
About the stored key -value stoted in the memory

The work is very interesting and amazing. I have a question about the number of frame's key-value stored in the memory and used as reference. The STM work stored all(or every 5 frames) the key-value of the past frames as reference. In SwiftNet, is it only need one key-value and only updated when the Variation is detected?

opened by mathmax12 0
About the motion model to generate the simulated images in pre-training

Thanks for sharing your amazing work! I have some questions about the pre-training stage. In Sec 4.2.1 of your paper, you mention that 'we maintain an implicit motion model to generate clips with length of 5.' This part seems different from the pre-training of STM. So what is the motion model and how does it works?

opened by blessingforever 0

Owner

haochen wang

GitHub

Real-Time-Student-Attendence-System - Real Time Student Attendence System

Real-Time-Student-Attendence-System The Student Attendance Management System Pro

1 Feb 15, 2022

Implement object segmentation on images using HOG algorithm proposed in CVPR 2005

HOG Algorithm Implementation Description HOG (Histograms of Oriented Gradients) Algorithm is an algorithm aiming to realize object segmentation (edge

2 Mar 12, 2022

Object tracking and object detection is applied to track golf puts in real time and display stats/games.

Putting_Game Object tracking and object detection is applied to track golf puts in real time and display stats/games. Works best with the Perfect Prac

1 Dec 29, 2021

Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation

STCN Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation Ho Kei Cheng, Yu-Wing Tai, Chi-Keung Tang [a

456 Dec 12, 2022

Using pytorch to implement unet network for liver image segmentation.

1 Dec 17, 2021

YOLOX-CondInst - Implement CondInst which is a instances segmentation method on YOLOX

YOLOX CondInst -- YOLOX 实例分割前言本项目是自己学习实例分割时，复现的代码. 通过自己编程，让自己对实例分割有更进一步的了解。若想

16 Nov 18, 2022

This Jupyter notebook shows one way to implement a simple first-order low-pass filter on sampled data in discrete time.

How to Implement a First-Order Low-Pass Filter in Discrete Time We often teach or learn about filters in continuous time, but then need to implement t

4 Aug 24, 2022

Real-time multi-object tracker using YOLO v5 and deep sort

This repository contains a two-stage-tracker. The detections generated by YOLOv5, a family of object detection architectures and models pretrained on the COCO dataset, are passed to a Deep Sort algorithm which tracks the objects. It can track any object that your Yolov5 model was trained to detect.

3.6k Jan 5, 2023

ICCV2021 Paper: AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection

107 Dec 20, 2022

Implementation for the paper 'YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs'

YOLO-ReT This is the original implementation of the paper: YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs. Prakhar Ganesh, Ya

69 Oct 19, 2022

Unofficial PyTorch implementation of "RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving" (ECCV 2020)

RTM3D-PyTorch The PyTorch Implementation of the paper: RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving (ECCV 2020

271 Nov 29, 2022

implement of SwiftNet:Real-time Video Object Segmentation

Related tags

Overview

SwiftNet

Requirements

Training

Inference

Performance

Acknowledgement

Citation

Comments

Resnet18 weights

Variation-Aware Trigger?

training pipeline

About the stored key -value stoted in the memory

About the motion model to generate the simulated images in pre-training

Owner

haochen wang

Real-Time-Student-Attendence-System - Real Time Student Attendence System

Implement object segmentation on images using HOG algorithm proposed in CVPR 2005

Object tracking and object detection is applied to track golf puts in real time and display stats/games.

Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation

Using pytorch to implement unet network for liver image segmentation.

YOLOX-CondInst - Implement CondInst which is a instances segmentation method on YOLOX

This Jupyter notebook shows one way to implement a simple first-order low-pass filter on sampled data in discrete time.

Real-time multi-object tracker using YOLO v5 and deep sort

ICCV2021 Paper: AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection

Implementation for the paper 'YOLO-ReT: Towards High Accuracy Real-time Object Detection on Edge GPUs'

Unofficial PyTorch implementation of "RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving" (ECCV 2020)

Lunar is a neural network aimbot that uses real-time object detection accelerated with CUDA on Nvidia GPUs.

Real Time Object Detection and Classification using Yolo Algorithm.

BED: A Real-Time Object Detection System for Edge Devices

Real-time Object Detection for Streaming Perception, CVPR 2022

Unofficial implementation of "TTNet: Real-time temporal and spatial video analysis of table tennis" (CVPR 2020)

Code for "NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video", CVPR 2021 oral

MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images

RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation