Super-BPD: Super Boundary-to-Pixel Direction for Fast Image Segmentation (CVPR 2020)

Overview

Super-BPD for Fast Image Segmentation (CVPR 2020)

Introduction

We propose direction-based super-BPD, an alternative to superpixel, for fast generic image segmentation, achieving state-of-the-art real-time result.

Citation

Please cite the related works in your publications if it helps your research:


@InProceedings{Wan_2020_CVPR,
author = {Wan, Jianqiang and Liu, Yang and Wei, Donglai and Bai, Xiang and Xu, Yongchao},
title = {Super-BPD: Super Boundary-to-Pixel Direction for Fast Image Segmentation},
booktitle = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2020}
}

Prerequisite

  • pytorch >= 1.3.0
  • g++ 7

Dataset

Testing

  • Compile cuda code for post-process.
cd post_process
python setup.py install
  • Download the pre-trained PascalContext model and put it in the saved folder.

  • Test the model and results will be saved in the test_pred_flux/PascalContext folder.

  • SEISM is used for evaluation of image segmentation.

Training

python train.py --dataset PascalContext
Issues
  • clarification on norm loss calculation; possible bug?

    clarification on norm loss calculation; possible bug?

    when i look at the image at https://github.com/JianqiangWan/Super-BPD/blob/master/post_process/2009_004607.png

    shown here image

    the norm_pred seems to decrease to blue (< 0.5) in the center of the cat's face (farther from the boundary). this also happen for all midpoints from the boundary of the cat. this is extremely different than the norm_gt

    when I look at the code in

    https://github.com/JianqiangWan/Super-BPD/blob/master/vis_flux.py#L45

    that seems like the correct calculation for the norm

    I've run this on a few other examples

    image

    and a similar thing seems to happen.

    this led me to go investigate the implementation of the loss

    If I'm understanding the loss as defined in the paper

    image

    that means norm_loss should be pred_flux - gt_flux like in https://github.com/JianqiangWan/Super-BPD/blob/master/train.py#L42

    norm_loss = weight_matrix * (pred_flux - gt_flux)**2
    

    however, this happens after https://github.com/JianqiangWan/Super-BPD/blob/master/train.py#L39. which, I believe, is incorrect

    I believe that L39 needs to happen after L42. otherwise, the norm_loss as-is is actually training the norm values to be angle values.

    This makes sense as if we look at the norm_pred outputs, they look more similar to the norm_angle outputs than they should be.

    HOWEVER, I could be completely misunderstanding the norm_loss term, so please let me know if I am! 🤞

    opened by rllin 6
  • Failed to compile the BPD module with errors under both Pytorch1.2 & Pytorch1.5

    Failed to compile the BPD module with errors under both Pytorch1.2 & Pytorch1.5

    Hi, Really nice work!

    We have tried to compile the BPD module in the post_process folder and got different error information based on Pytorch1.2 or Pytorch1.5 as below.

    • Pytorch1.2 errors:

    image

    • Pytorch1.5 errors: image

    Thanks!

    opened by PkuRainBow 1
  • How to test a single RGB image?

    How to test a single RGB image?

    Thank you very much for your work. If I want to test an RGB image, can I get a semantic segmentation graph through the model input picture, and I want clear visualization results, can I get a more precise semantic segmentation graph?

    opened by carfei 1
  • Release of the Training code & scripts

    Release of the Training code & scripts

    Dear Authors,

    Really excellent work~ I am wondering about your plan on releasing the training code & scripts.

    Regards

    opened by PkuRainBow 0
  • 请问二维数组flux变量中存储的是图片的什么信息?

    请问二维数组flux变量中存储的是图片的什么信息?

    请问二维数组flux变量中存储的是图片的什么信息?里面的数字代表什么意思?谢谢。

    opened by tangjiaxi98 0
  • Could you explain the output data meaning?

    Could you explain the output data meaning?

    The point is that the model works fast but bpd_cuda.forward call takes a large amount of time so the question is: what output data is and can I not to call bpd_cuda.forward to get data about segments?

    Also is the network orientied for specific domain image content? Cause for my test images it results bad.

    Original image 716c971s-960

    Visualization grid result

    root root

    super_BPDs super_BPDs

    super_BPDs_before_dilation super_BPDs_before_dilation

    super_BPDs_after_dilation super_BPDs_after_dilation

    my code to inference single image:

    model = VGG16()
    model.load_state_dict(torch.load('/home/algernone/git_projects/Super-BPD/saved/PascalContext_400000.pth'))
    
    model.eval()
    model.cuda()
    
    
    image_path = '/home/algernone/test_imgs/716c971s-960.jpg'
    image = cv2.imread(image_path, 1)
    src_img = image.copy()
    height, width = image.shape[:2]
    image = image.astype(np.float32)
    image -= IMAGE_MEAN
    image = image.transpose(2, 0, 1)
    image = image[np.newaxis]
    image = torch.from_numpy(image)
    
    tik = time()
    pred_flux = model(image.cuda())
    flux = pred_flux.data[0, ...]
    
    vis_flux(src_img, flux)
    
    angles = torch.atan2(flux[1,...], flux[0,...]) 
    angles[angles < 0] += 2*math.pi 
    
    height, width = angles.shape 
    
    # unit: degree 
    # theta_a, theta_l, theta_s, S_o, 45, 116, 68, 5 
    results = bpd_cuda.forward(angles, height, width, 45, 116, 68, 5) 
    root_points, super_BPDs_before_dilation, super_BPDs_after_dilation, super_BPDs = results 
    
    
    root_points = root_points.cpu().numpy()
    super_BPDs_before_dilation = super_BPDs_before_dilation.cpu().numpy()
    super_BPDs_after_dilation = super_BPDs_after_dilation.cpu().numpy()
    super_BPDs = super_BPDs.cpu().numpy()
    
    cv2.imwrite('root.png', 255*(root_points > 0))
    cv2.imwrite('super_BPDs.png', label2color(super_BPDs))
    cv2.imwrite('super_BPDs_before_dilation.png', label2color(super_BPDs_before_dilation))
    cv2.imwrite('super_BPDs_after_dilation.png', label2color(super_BPDs_after_dilation))
    
    opened by chamecall 1
  • cl.exe' failed with exit status 2

    cl.exe' failed with exit status 2

    Hi! When running the code ”python setup.py install“, there is something wrong like picture below. How can i deal with it.

    QQ截图20201213062646

    opened by liqier 1
  • Why batchsize is 1 by default

    Why batchsize is 1 by default

    Thank you for your code! We know that generally larger batchsize is more beneficial to the result,I have seen that the batchsize of many open source code for split tasks is set to 1 by default, but why?Looking forward to your reply!

    opened by darknli 1
  • RuntimeError: CUDA error: invalid device function

    RuntimeError: CUDA error: invalid device function

    Thank you very much for your work.

    1. I compiled and installed the post_process program.(success)

    2. I want to run demo to reproduce the result but got error

      results = bpd_cuda.forward(angles, height, width, 45, 116, 68, 5) RuntimeError: CUDA error: invalid device function

    Here is my torch version torch 1.6.0 torchvision 0.7.0

    I run the program in V100 and had 4GPUs Thank you very much if you can reply.

    opened by zyybutter 2
Owner
Master Student of VLR Group, Hust
null
Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation, CVPR 2018

Learning Pixel-level Semantic Affinity with Image-level Supervision This code is deprecated. Please see https://github.com/jiwoon-ahn/irn instead. Int

Jiwoon Ahn 301 Jan 10, 2022
The implementation of ICASSP 2020 paper "Pixel-level self-paced learning for super-resolution"

Pixel-level Self-Paced Learning for Super-Resolution This is an official implementaion of the paper Pixel-level Self-Paced Learning for Super-Resoluti

Elon Lin 39 Nov 25, 2021
Implementation of CVPR 2020 Dual Super-Resolution Learning for Semantic Segmentation

Dual super-resolution learning for semantic segmentation 2021-01-02 Subpixel Update Happy new year! The 2020-12-29 update of SISR with subpixel conv p

Sam 74 Jan 4, 2022
Boundary-preserving Mask R-CNN (ECCV 2020)

BMaskR-CNN This code is developed on Detectron2 Boundary-preserving Mask R-CNN ECCV 2020 Tianheng Cheng, Xinggang Wang, Lichao Huang, Wenyu Liu Video

Hust Visual Learning Team 160 Jan 11, 2022
Exploring Cross-Image Pixel Contrast for Semantic Segmentation

Exploring Cross-Image Pixel Contrast for Semantic Segmentation Exploring Cross-Image Pixel Contrast for Semantic Segmentation, Wenguan Wang, Tianfei Z

Tianfei Zhou 351 Jan 16, 2022
code for `Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation`

Look Closer to Segment Better: Boundary Patch Refinement for Instance Segmentation (CVPR 2021) Introduction PBR is a conceptually simple yet effective

H.Chen 101 Jan 12, 2022
Generic Event Boundary Detection: A Benchmark for Event Segmentation

Generic Event Boundary Detection: A Benchmark for Event Segmentation We release our data annotation & baseline codes for detecting generic event bound

null 30 Jan 6, 2022
Code for Boundary-Aware Segmentation Network for Mobile and Web Applications

BASNet Boundary-Aware Segmentation Network for Mobile and Web Applications This repository contain implementation of BASNet in tensorflow/keras. comme

Hamid Ali 3 Sep 27, 2021
[AAAI-2021] Visual Boundary Knowledge Translation for Foreground Segmentation

Trans-Net Code for (Visual Boundary Knowledge Translation for Foreground Segmentation, AAAI2021). [https://ojs.aaai.org/index.php/AAAI/article/view/16

ZJU-VIPA 1 Nov 10, 2021
An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

Kakao Brain 14 Jan 18, 2022
Official respository for "Modeling Defocus-Disparity in Dual-Pixel Sensors", ICCP 2020

Official respository for "Modeling Defocus-Disparity in Dual-Pixel Sensors", ICCP 2020 BibTeX @INPROCEEDINGS{punnappurath2020modeling, author={Abhi

Abhijith Punnappurath 14 Jul 30, 2021
git《Investigating Loss Functions for Extreme Super-Resolution》(CVPR 2020) GitHub:

Investigating Loss Functions for Extreme Super-Resolution NTIRE 2020 Perceptual Extreme Super-Resolution Submission. Our method ranked first and secon

Sejong Yang 101 Dec 11, 2021
Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning, CVPR 2021

Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning By Zhenda Xie*, Yutong Lin*, Zheng Zhang, Yue Ca

Zhenda Xie 233 Jan 7, 2022
Code for "PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation" CVPR 2019 oral

Good news! We release a clean version of PVNet: clean-pvnet, including how to train the PVNet on the custom dataset. Use PVNet with a detector. The tr

ZJU3DV 657 Jan 1, 2022
Per-Pixel Classification is Not All You Need for Semantic Segmentation

MaskFormer: Per-Pixel Classification is Not All You Need for Semantic Segmentation Bowen Cheng, Alexander G. Schwing, Alexander Kirillov [arXiv] [Proj

Facebook Research 781 Jan 11, 2022
Pytorch Implementation for NeurIPS (oral) paper: Pixel Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Pixel-Level Cycle Association This is the Pytorch implementation of our NeurIPS 2020 Oral paper Pixel-Level Cycle Association: A New Perspective for D

null 82 Jan 7, 2022
Pixel-wise segmentation on VOC2012 dataset using pytorch.

PiWiSe Pixel-wise segmentation on the VOC2012 dataset using pytorch. FCN SegNet PSPNet UNet RefineNet For a more complete implementation of segmentati

Bodo Kaiser 370 Jan 13, 2022
Retinal Vessel Segmentation with Pixel-wise Adaptive Filters (ISBI 2022)

Retinal Vessel Segmentation with Pixel-wise Adaptive Filters (ISBI 2022) Introdu

anonymous 2 Jan 11, 2022
PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network"

HAN PyTorch code for our ECCV 2020 paper "Single Image Super-Resolution via a Holistic Attention Network" This repository is for HAN introduced in the

五维空间 114 Jan 14, 2022