Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Related tags

Deep Learning HMMN
Overview

Hierarchical Memory Matching Network for Video Object Segmentation

Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, Euntai Kim

ICCV 2021

no_image

This is the implementation of HMMN.
This code is based on STM (ICCV 2019): [link].
Please see our paper for the details: [paper]

Hierarchical Memory Matching Network for Video Object Segmentation (ICCV 2021)

Dependencies

  • Python 3.8
  • PyTorch 1.8.1
  • numpy, opencv, pillow

Trained model

  • Download pre-trained weights into the same folder with demo scripts
    Link: [weights]

Code

  • DAVIS-2016 validation set (Single-object)
python eval_DAVIS.py -g '0' -s val -y 16 -D [path/to/DAVIS]
  • DAVIS-2017 validation set (Multi-object)
python eval_DAVIS.py -g '0' -s val -y 17 -D [path/to/DAVIS]

Pre-computed Results

We also provide pre-computed results for benchmark sets.

Bibtex

@inproceedings{seong2021hierarchical,
  title={Hierarchical Memory Matching Network for Video Object Segmentation},
  author={Seong, Hongje and Oh, Seoung Wug and Lee, Joon-Young and Lee, Seongwon and Lee, Suhyeon and Kim, Euntai},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2021}
}

Terms of Use

This software is for non-commercial use only. The source code is released under the Attribution-NonCommercial-ShareAlike (CC BY-NC-SA) Licence (see this for details)

You might also like...
Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

NuPIC Numenta Platform for Intelligent Computing The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implem

Implementation of Hierarchical Transformer Memory (HTM) for Pytorch
Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Hierarchical Transformer Memory (HTM) - Pytorch Implementation of Hierarchical Transformer Memory (HTM) for Pytorch. This Deepmind paper proposes a si

A Pytorch implementation of
A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generative Modeling" (ICCV 2021)

Manifold Matching via Deep Metric Learning for Generative Modeling A Pytorch implementation of "Manifold Matching via Deep Metric Learning for Generat

Video Instance Segmentation with a Propose-Reduce Paradigm (ICCV 2021)
Video Instance Segmentation with a Propose-Reduce Paradigm (ICCV 2021)

Propose-Reduce VIS This repo contains the official implementation for the paper: Video Instance Segmentation with a Propose-Reduce Paradigm Huaijia Li

Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021
Implementation of Memory-Efficient Neural Networks with Multi-Level Generation, ICCV 2021

Memory-Efficient Multi-Level In-Situ Generation (MLG) By Jiaqi Gu, Hanqing Zhu, Chenghao Feng, Mingjie Liu, Zixuan Jiang, Ray T. Chen and David Z. Pan

Code for C2-Matching (CVPR2021). Paper: Robust Reference-based Super-Resolution via C2-Matching.
Code for C2-Matching (CVPR2021). Paper: Robust Reference-based Super-Resolution via C2-Matching.

C2-Matching (CVPR2021) This repository contains the implementation of the following paper: Robust Reference-based Super-Resolution via C2-Matching Yum

A Python implementation of the Locality Preserving Matching (LPM) method for pruning outliers in image matching.

LPM_Python A Python implementation of the Locality Preserving Matching (LPM) method for pruning outliers in image matching. The code is established ac

Segcache: a memory-efficient and scalable in-memory key-value cache for small objects

Segcache: a memory-efficient and scalable in-memory key-value cache for small objects This repo contains the code of Segcache described in the followi

Episodic-memory - Ego4D Episodic Memory Benchmark

Ego4D Episodic Memory Benchmark EGO4D is the world's largest egocentric (first p

Comments
  • Regarding the results on YouTube-VOS

    Regarding the results on YouTube-VOS

    Hi,

    THX for releasing the inference code of DAVIS.

    May I ask what frame rate did you use in the evaluation of HMMN and KMN on YouTube-VOS? Validation or Validation All Frames? 6FPS or 30FPS?

    Best.

    opened by z-x-yang 2
  • question about the top-k guided memory matching

    question about the top-k guided memory matching

    thanks for your sharing . wonderful job, but why i didn't see the dropout in top-k guided memory matching and the conv3*3 before it in your code. and the KV_Q_r3 = KeyValue(512, keydim=128, valdim=-1, only_key=True) means there is no need for values. What do I understand it ?

    opened by mmyjjl1009 2
  • Dropout in the top-k guided memory matching module and the value of query(r2,r3)

    Dropout in the top-k guided memory matching module and the value of query(r2,r3)

    Excellent work! I have some question about the dropout in the top-k guided memory matching module and the value of query(r2,r3). In your code, you doesn't use the value of query of r2 and r3 stage, but in the paper you use the value to generate Z3. The operation in the code is different from the description in the paper.

    opened by LingyiHongfd 2
  • How to train your model?

    How to train your model?

    I noticed that this code only included evaluation part and lack the training part. It would be really helpful if you could proide the training code. Another question is that the model in this code has some differences compared to the paper. In the top-k guided memory matching module, the model in this code don't have the VQ3 add to the result which come out from dropout. Instead, it directly uses results after the second ''matmul'' as the output of the top-k module (in Figure.6 in the paper this is Z4). Could you please explain the reason? Moreover, I wonder if I need to add a dropout layer, a conv3 layer and a ''VQ3 add.'' layer when training the model. Of course if you can provide me with your training code, it would be most helpful! Thank you for your time! 6650b00feaa8005421926ba4506cbe5

    opened by hescluke 0
Owner
Hongje Seong
Hongje Seong
Implementation of "Efficient Regional Memory Network for Video Object Segmentation" (Xie et al., CVPR 2021).

RMNet This repository contains the source code for the paper Efficient Regional Memory Network for Video Object Segmentation. Cite this work @inprocee

Haozhe Xie 76 Dec 14, 2022
The Dual Memory is build from a simple CNN for the deep memory and Linear Regression fro the fast Memory

Simple-DMA a simple Dual Memory Architecture for classifications. based on the paper Dual-Memory Deep Learning Architectures for Lifelong Learning of

null 1 Jan 27, 2022
LBK 20 Dec 2, 2022
[2021][ICCV][FSNet] Full-Duplex Strategy for Video Object Segmentation

Full-Duplex Strategy for Video Object Segmentation (ICCV, 2021) Authors: Ge-Peng Ji, Keren Fu, Zhe Wu, Deng-Ping Fan*, Jianbing Shen, & Ling Shao This

Daniel-Ji 55 Dec 22, 2022
Official implementation of the ICCV 2021 paper "Joint Inductive and Transductive Learning for Video Object Segmentation"

JOINT This is the official implementation of Joint Inductive and Transductive learning for Video Object Segmentation, to appear in ICCV 2021. @inproce

Yunyao 35 Oct 16, 2022
Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation

STCN Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation Ho Kei Cheng, Yu-Wing Tai, Chi-Keung Tang [a

Rex Cheng 456 Dec 12, 2022
PyTorch Code of "Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamics"

Memory In Memory Networks It is based on the paper Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spati

Yang Li 12 May 30, 2022
This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows" on Object Detection and Instance Segmentation.

Swin Transformer for Object Detection This repo contains the supported code and configuration files to reproduce object detection results of Swin Tran

Swin Transformer 1.4k Dec 30, 2022
Codes of paper "Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling"

Unseen Object Amodal Instance Segmentation (UOAIS) Seunghyeok Back, Joosoon Lee, Taewon Kim, Sangjun Noh, Raeyoung Kang, Seongho Bak, Kyoobin Lee This

GIST-AILAB 92 Dec 13, 2022
Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

NuPIC Numenta Platform for Intelligent Computing The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implem

Numenta 6.3k Dec 30, 2022