FaPN: Feature-aligned Pyramid Network for Dense Image Prediction [arXiv] [Project Page]
@inproceedings{
huang2021fapn,
title={{FaPN}: Feature-aligned Pyramid Network for Dense Image Prediction},
author={Shihua Huang and Zhichao Lu and Ran Cheng and Cheng He},
booktitle={International Conference on Computer Vision (ICCV)},
year={2021}
}
Overview
| FaPN vs. FPN | Before vs. After Alignment |
|---|---|
 |
 |
This project provides the official implementation for our ICCV2021 paper "FaPN: Feature-aligned Pyramid Network for Dense Image Prediction" based on Detectron2. FaPN is a simple yet effective top-down pyramidal architecture to generate multi-scale features for dense image prediction. Comprised of a feature alignment module (FAM) and a feature selection module (FSM), FaPN addresses the issue of feature alignment in the original FPN, leading to substaintial improvements on various dense prediction tasks, such as object detection, semantic, instance, panoptic segmentation, etc.
Installation
This project is based on Detectron2, which can be constructed as follows.
- Install Detectron2 following the instructions.
- Setup the dataset following the structure.
- Copy this project to
/path/to/detectron2 - Install DCNv2 following Install DCNv2.md.
Training
To train a model with 8 GPUs, run:
cd /path/to/detectron2/tools
python3 train_net.py --config-file <config.yaml> --num-gpus 8
For example, to launch Faster R-CNN training (1x schedule) with ResNet-50 backbone on 8 GPUs, one should execute:
cd /path/to/detectron2/tools
python3 train_net.py --config-file ../configs\COCO-Detection\faster_rcnn_R_50_FAN_1x.yaml --num-gpus 8
Evaluation
To evaluate a pre-trained model with 8 GPUs, run:
cd /path/to/detectron2/tools
python3 train_net.py --config-file <config.yaml> --num-gpus 8 --eval-only MODEL.WEIGHTS /path/to/model_checkpoint
Results
COCO Object Detection
Faster R-CNN + FaPN:
| Name | lr sched |
box AP |
box APs |
box APm |
box APl |
download |
|---|---|---|---|---|---|---|
| R50 | 1x | 39.2 | 24.5 | 43.3 | 49.1 | model | log |
| R101 | 3x | 42.8 | 27.0 | 46.2 | 54.9 | model | log |
Cityscapes Semantic Segmentation
PointRend + FaPN:
| Name | lr sched |
mask mIoU |
mask i_IoU |
mask IoU_sup |
mask iIoU_sup |
download |
|---|---|---|---|---|---|---|
| R50 | 1x | 80.0 | 61.3 | 90.6 | 78.5 | model | log |
| R101 | 1x | 80.1 | 62.2 | 90.8 | 78.6 | model | log |
COCO Instance Segmentation
Mask R-CNN + FaPN:
| Name | lr sched |
mask AP |
mask APs |
box AP |
box APs |
download |
|---|---|---|---|---|---|---|
| R50 | 1x | 36.4 | 18.1 | 39.8 | 24.3 | model | log |
| R101 | 3x | 39.4 | 20.9 | 43.8 | 27.4 | model | log |
PointRend + FaPN:
| Name | lr sched |
mask AP |
mask APs |
box AP |
box APs |
download |
|---|---|---|---|---|---|---|
| R50 | 1x | 37.6 | 18.6 | 39.4 | 24.2 | model | log |
COCO Panoptic Segmentation
PanopticFPN + FaPN:
| Name | lr sched |
PQ | mask mIoU |
St PQ |
box AP |
Th PQ |
download |
|---|---|---|---|---|---|---|---|
| R50 | 1x | 41.1 | 43.4 | 32.5 | 38.7 | 46.9 | model | log |
| R101 | 3x | 44.2 | 45.7 | 35.0 | 43.0 | 53.3 | model | log |
35 Dec 5, 2022
53 Dec 20, 2022
110 Dec 20, 2022
9 Jul 30, 2022
912 Dec 21, 2022
29 Aug 23, 2022
50 Dec 21, 2022
32 Oct 26, 2022
3 Apr 16, 2022
69 Dec 15, 2022
33 Nov 1, 2022
41 Oct 21, 2022
1.3k Dec 28, 2022
244 Dec 27, 2022
403 Dec 13, 2022
272 Dec 20, 2022
115 Dec 23, 2022