DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision
Paper | Project page | Demo (Youtube) | Demo (Bilibili)
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.
Shiyi Lan, Zhiding Yu, Chris Choy, Subhashree Radhakrishnan, Guilin Liu, Yuke Zhu, Larry Davis, Anima Anandkumar
International Conference on Computer Vision (ICCV) 2021
This repository contains the official Pytorch implementation of training & evaluation code and pretrained models for DiscoBox. DiscoBox is a state of the art framework that can jointly predict high quality instance segmentation and semantic correspondence from box annotations.
We use MMDetection v2.10.0 as the codebase.
All of our models are trained and tested using automatic mixed precision, which leverages float16 for speedup and less GPU memory consumption.
Installation
This implementation is based on PyTorch==1.9.0, mmcv==2.13.0, and mmdetection==2.10.0
Please refer to get_started.md for installation.
Or you can download the docker image from our dockerhub repository.
Models
Results on COCO val 2017
| Backbone | Weights | AP | AP@50 | AP@75 | AP@Small | AP@Medium | AP@Large |
|---|---|---|---|---|---|---|---|
| ResNet-50 | download | 30.7 | 52.6 | 30.6 | 13.3 | 34.1 | 45.6 |
| ResNet-101-DCN | download | 35.3 | 59.1 | 35.4 | 16.9 | 39.2 | 53.0 |
| ResNeXt-101-DCN | download | 37.3 | 60.4 | 39.1 | 17.8 | 41.1 | 55.4 |
Results on COCO test-dev
We also evaluate the models in the section Results on COCO val 2017 with the same weights on COCO test-dev.
| Backbone | Weights | AP | AP@50 | AP@75 | AP@Small | AP@Medium | AP@Large |
|---|---|---|---|---|---|---|---|
| ResNet-50 | download | 32.0 | 53.6 | 32.6 | 11.7 | 33.7 | 48.4 |
| ResNet-101-DCN | download | 35.8 | 59.8 | 36.4 | 16.9 | 38.7 | 52.1 |
| ResNeXt-101-DCN | download | 37.9 | 61.4 | 40.0 | 18.0 | 41.1 | 53.9 |
Training
COCO
ResNet-50 (8 GPUs):
bash tools/dist_train.sh \
configs/discobox/discobox_solov2_r50_fpn_3x.py 8
ResNet-101-DCN (8 GPUs):
bash tools/dist_train.sh \
configs/discobox/discobox_solov2_r101_dcn_fpn_3x.py 8
ResNeXt-101-DCN (8 GPUs):
bash tools/dist_train.sh \
configs/discobox/discobox_solov2_x101_dcn_fpn_3x.py 8
Pascal VOC 2012
ResNet-50 (4 GPUs):
bash tools/dist_train.sh \
configs/discobox/discobox_solov2_voc_r50_fpn_6x.py 4
ResNet-101 (4 GPUs):
bash tools/dist_train.sh \
configs/discobox/discobox_solov2_voc_r101_fpn_6x.py 4
Testing
COCO
ResNet-50 (8 GPUs):
bash tools/dist_test.sh \
configs/discobox/discobox_solov2_r50_fpn_3x.py \
work_dirs/coco_r50_fpn_3x.pth 8 --eval segm
ResNet-101-DCN (8 GPUs):
bash tools/dist_test.sh \
configs/discobox/discobox_solov2_r101_dcn_fpn_3x.py \
work_dirs/coco_r101_dcn_fpn_3x.pth 8 --eval segm
ResNeXt-101-DCN (GPUs):
bash tools/dist_test.sh \
configs/discobox/discobox_solov2_x101_dcn_fpn_3x_fp16.py \
work_dirs/coco_x101_dcn_fpn_3x.pth 8 --eval segm
Pascal VOC 2012 (COCO API)
ResNet-50 (4 GPUs):
bash tools/dist_test.sh \
configs/discobox/discobox_solov2_voc_r50_fpn_3x_fp16.py \
work_dirs/voc_r50_6x.pth 4 --eval segm
ResNet-101 (4 GPUs):
bash tools/dist_test.sh \
configs/discobox/discobox_solov2_voc_r101_fpn_3x_fp16.py \
work_dirs/voc_r101_6x.pth 4 --eval segm
Pascal VOC 2012 (Matlab)
Step 1: generate results
ResNet-50 (4 GPUs):
bash tools/dist_test.sh \
configs/discobox/discobox_solov2_voc_r50_fpn_3x_fp16.py \
work_dirs/voc_r50_6x.pth 4 \
--format-only \
--options "jsonfile_prefix=work_dirs/voc_r50_results.json"
ResNet-101 (4 GPUs):
bash tools/dist_test.sh \
configs/discobox/discobox_solov2_voc_r101_fpn_3x_fp16.py \
work_dirs/voc_r101_6x.pth 4 \
--format-only \
--options "jsonfile_prefix=work_dirs/voc_r101_results.json"
Step 2: format conversion
ResNet-50:
python tools/json2mat.pywork_dirs/voc_r50_results.json work_dirs/voc_r50_results.mat
ResNet-101:
python tools/json2mat.pywork_dirs/voc_r101_results.json work_dirs/voc_r101_results.mat
Step 3: evaluation
Please visit BBTP for the evaluation code written in Matlab.
PF-Pascal
Please visit this repository.
LICENSE
Please check the LICENSE file. DiscoBox may be used non-commercially, meaning for research or evaluation purposes only. For business inquiries, please contact [email protected].
Citation
@article{lan2021discobox,
title={DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision},
author={Lan, Shiyi and Yu, Zhiding and Choy, Christopher and Radhakrishnan, Subhashree and Liu, Guilin and Zhu, Yuke and Davis, Larry S and Anandkumar, Anima},
journal={arXiv preprint arXiv:2105.06464},
year={2021}
}
3 Oct 6, 2022
49 Nov 23, 2022
3.2k Dec 30, 2022
443 Dec 6, 2022
364 Dec 14, 2022
111 Dec 31, 2022
28 Nov 16, 2022
35 Dec 6, 2022
87 Dec 30, 2022
11.3k Dec 30, 2022
34 Jan 19, 2022
307 Jan 3, 2023
217 Jan 4, 2023
123 Dec 27, 2022
173 Dec 21, 2022
489 Jan 7, 2023
153 Dec 14, 2022
97 Dec 23, 2022