A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation

Related tags

Deep Learning Amodal-Instance-Seg-ASBU
Overview

Paper

Khoi Nguyen, Sinisa Todorovic "A Weakly Supervised Amodal Segmenter with Boundary Uncertainty Estimation", accepted to ICCV 2021

Our code is mainly based on the code from the paper: Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, Chen Change Loy, "Self-Supervised Scene De-occlusion"

Requirements

  • pytorch>=0.4.1

    pip install -r requirements.txt

Data Preparation

COCOA dataset proposed in Semantic Amodal Segmentation.

  1. Download COCO2014 train and val images from here and unzip.

  2. Download COCOA annotations from here and untar.

  3. Ensure the COCOA folder looks like:

    COCOA/
  |-- train2014/
  |-- val2014/
  |-- annotations/
    |-- COCO_amodal_train2014.json
    |-- COCO_amodal_val2014.json
    |-- COCO_amodal_test2014.json
    |-- ...

  4. Create symbolic link:

    cd deocclusion
mkdir data
cd data
ln -s /path/to/COCOA

KINS dataset proposed in Amodal Instance Segmentation with KINS Dataset.

  1. Download left color images of object data in KITTI dataset from here and unzip.

  2. Download KINS annotations from here corresponding to this commit.

  3. Ensure the KINS folder looks like:

    KINS/
  |-- training/image_2/
  |-- testing/image_2/
  |-- instances_train.json
  |-- instances_val.json

  4. Create symbolic link:

    cd deocclusion/data
ln -s /path/to/KINS

Train

train PCNet-M

  1. Train (taking COCOA for example).

    ./train_pcnet_m_std_no_rgb_gaussian.sh

  2. Monitoring status and visual results using tensorboard.

    sh tensorboard.sh $PORT

Evaluate

  • Execute:

    ./test_pcnet_m.sh

Bibtex

@InProceedings{Nguyen_2021_ICCV,
    author    = {Nguyen, Khoi and Todorovic, Sinisa},
    title     = {A Weakly Supervised Amodal Segmenter With Boundary Uncertainty Estimation},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {7396-7405}
}

Acknowledgement

  1. We developed our approach based on the code from https://github.com/XiaohangZhan/deocclusion/

  2. We used the code and models of GCA-Matting in our demo.

  3. We modified some code from pytorch-inpainting-with-partial-conv to train the PCNet-C.

You might also like...
Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)
Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation (RA-L/ICRA 2020)

Aerial Depth Completion This work is described in the letter "Aerial Single-View Depth Completion with Image-Guided Uncertainty Estimation", by Lucas

ETHZ V4RL 70 Dec 22, 2022
Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation
Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation

Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation

Bae, Gwangbin 95 Jan 4, 2023
Face uncertainty quantification or estimation using PyTorch.

Face-uncertainty-pytorch This is a demo code of face uncertainty quantification or estimation using PyTorch. The uncertainty of face recognition is af

Kaen 3 Sep 16, 2022
Symmetry and Uncertainty-Aware Object SLAM for 6DoF Object Pose Estimation
Symmetry and Uncertainty-Aware Object SLAM for 6DoF Object Pose Estimation

SUO-SLAM This repository hosts the code for our CVPR 2022 paper "Symmetry and Uncertainty-Aware Object SLAM for 6DoF Object Pose Estimation". ArXiv li

Robot Perception & Navigation Group (RPNG) 97 Jan 3, 2023
noisy labels; missing labels; semi-supervised learning; entropy; uncertainty; robustness and generalisation.

ProSelfLC: CVPR 2021 ProSelfLC: Progressive Self Label Correction for Training Robust Deep Neural Networks For any specific discussion or potential fu

amos_xwang 57 Dec 4, 2022
Data-Uncertainty Guided Multi-Phase Learning for Semi-supervised Object Detection

An official implementation of paper Data-Uncertainty Guided Multi-Phase Learning for Semi-supervised Object Detection

null 11 Nov 23, 2022
Weakly Supervised Learning of Rigid 3D Scene Flow
Weakly Supervised Learning of Rigid 3D Scene Flow

Weakly Supervised Learning of Rigid 3D Scene Flow This repository provides code and data to train and evaluate a weakly supervised method for rigid 3D

Zan Gojcic 124 Dec 27, 2022
Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation

Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation This paper has been accepted and early accessed

Yun Liu 39 Sep 20, 2022
A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains (IJCV submission)
A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains (IJCV submission)

wsss-analysis The code of: A Comprehensive Analysis of Weakly-Supervised Semantic Segmentation in Different Image Domains, arXiv pre-print 2019 paper.

Lyndon Chan 48 Dec 18, 2022
Comments
  • ModuleNotFoundError: No module named 'mmdet.ops'

    ModuleNotFoundError: No module named 'mmdet.ops'

    Hello! The "mmdet" module which should be installed is not written in the "requirements.txt".And I install it but it occurs error that I can not solve it. Could you tell me what version of your "mmcv" and "mmdet" module? Thank you. The log: Traceback (most recent call last): File "main.py", line 9, in from trainer import Trainer File "/home/user15/Amodal-Instance-Seg-ASBU/trainer.py", line 12, in import models File "/home/user15/Amodal-Instance-Seg-ASBU/models/init.py", line 1, in from . import backbone File "/home/user15/Amodal-Instance-Seg-ASBU/models/backbone/init.py", line 1, in from .unet import * File "/home/user15/Amodal-Instance-Seg-ASBU/models/backbone/unet/init.py", line 1, in from .unet_model import * File "/home/user15/Amodal-Instance-Seg-ASBU/models/backbone/unet/unet_model.py", line 8, in from mmdet.ops.dcn.deform_conv import ModulatedDeformConvPack as Deform ModuleNotFoundError: No module named 'mmdet.ops' Killing subprocess 39673 Killing subprocess 39674 Traceback (most recent call last): File "/home/user15/anaconda3/envs/occlusion/lib/python3.7/runpy.py", line 193, in _run_module_as_main "main", mod_spec) File "/home/user15/anaconda3/envs/occlusion/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/user15/anaconda3/envs/occlusion/lib/python3.7/site-packages/torch/distributed/launch.py", line 340, in main() File "/home/user15/anaconda3/envs/occlusion/lib/python3.7/site-packages/torch/distributed/launch.py", line 326, in main sigkill_handler(signal.SIGTERM, None) # not coming back File "/home/user15/anaconda3/envs/occlusion/lib/python3.7/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd) subprocess.CalledProcessError: Command '['/home/user15/anaconda3/envs/occlusion/bin/python', '-u', 'main.py', '--local_rank=1', '--config', 'experiments/KINS/pcnet_m/config_train_default_no_rgb.yaml', '--launcher', 'pytorch', '--exp_path', 'experiments/KINS/pcnet_m_default_no_rgb']' returned non-zero exit status 1.

    opened by landian60 1
  • Reproduce results in your paper

    Reproduce results in your paper

    Hello! Thanks for your wonderful work.

    I have trained your model following the github instructions ./train_pcnet_m_std_no_rgb_gaussian.sh, and tried to calculate the mIOU results on COCOA-test using your code. But I got 88.41 rather than 89.87 as you stated in your paper. Could you please let me know how to get that result? Is it correct for me to use ./train_pcnet_m_std_no_rgb_gaussian.sh which uses the config file of config_train_std_no_rgb_gaussian.yaml?

    opened by Championchess 0
Owner
Khoi Nguyen
Ph.D. in Computer Science with major in Computer Vision
Khoi Nguyen
GitHub
Codes of paper "Unseen Object Amodal Instance Segmentation via Hierarchical Occlusion Modeling"

Unseen Object Amodal Instance Segmentation (UOAIS) Seunghyeok Back, Joosoon Lee, Taewon Kim, Sangjun Noh, Raeyoung Kang, Seongho Bak, Kyoobin Lee This

GIST-AILAB 92 Dec 13, 2022
Codebase for Amodal Segmentation through Out-of-Task andOut-of-Distribution Generalization with a Bayesian Model

Codebase for Amodal Segmentation through Out-of-Task andOut-of-Distribution Generalization with a Bayesian Model

Yihong Sun 12 Nov 15, 2022
Finding an Unsupervised Image Segmenter in each of your Deep Generative Models

Finding an Unsupervised Image Segmenter in each of your Deep Generative Models Description Recent research has shown that numerous human-interpretable

Luke Melas-Kyriazi 61 Oct 17, 2022
Official PyTorch implementation of Segmenter: Transformer for Semantic Segmentation

Segmenter: Transformer for Semantic Segmentation Segmenter: Transformer for Semantic Segmentation by Robin Strudel*, Ricardo Garcia*, Ivan Laptev and

null 594 Jan 6, 2023
"Segmenter: Transformer for Semantic Segmentation" reproduced via mmsegmentation

Segmenter-based-on-OpenMMLab "Segmenter: Transformer for Semantic Segmentation, arxiv 2105.05633." reproduced via mmsegmentation. We reproduce Segment

EricKani 22 Feb 24, 2022
The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter

FAPIS The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter Introduction This repo is primari

Khoi Nguyen 8 Dec 11, 2022
Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

Hybrid-Supervised Object Detection System Object detection system trained by hybrid-supervision/weakly semi-supervision (HSOD/WSSOD): This project is

null 5 Dec 10, 2022
Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Noise Contrastive Estimation for pyTorch Overview This repository contains a re-implementation of the Noise Contrastive Estimation algorithm, implemen

Denis Emelin 42 Nov 24, 2022
An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

Kakao Brain 72 Dec 28, 2022
git git《Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking》(CVPR 2021) GitHub:git2] 《Masksembles for Uncertainty Estimation》(CVPR 2021) GitHub:git3]

Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li Accepted by CVPR

NingWang 236 Dec 22, 2022