[ICCV 2021] FaPN: Feature-aligned Pyramid Network for Dense Image Prediction

Related tags

Deep Learning semantic-segmantation object-detecting panoptic-segmentation real-time-semantic-segmentation instance-segmenation feature-alignment
Overview

FaPN: Feature-aligned Pyramid Network for Dense Image Prediction [arXiv] [Project Page]

@inproceedings{
  huang2021fapn,
  title={{FaPN}: Feature-aligned Pyramid Network for Dense Image Prediction},
  author={Shihua Huang and Zhichao Lu and Ran Cheng and Cheng He},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2021}
}

Overview

FaPN vs. FPN Before vs. After Alignment

This project provides the whole official implementation for our ICCV2021 paper "FaPN: Feature-aligned Pyramid Network for Dense Image Prediction" based on Detectron2, PanoticFCN, and MaskFormer. FaPN is a simple yet effective top-down pyramidal architecture to generate multi-scale features for dense image prediction. Comprised of a feature alignment module (FAM) and a feature selection module (FSM), FaPN addresses the issue of feature alignment in the original FPN, leading to substaintial improvements on various dense prediction tasks, such as object detection, semantic, instance, panoptic segmentation, etc.

Installation

This project is based on Detectron2, which can be constructed as follows.

Results

COCO Object Detection

Faster R-CNN + FaPN:

Name lr
sched		 box
AP		 box
APs		 box
APm		 box
APl		 download
R50 1x 39.2 24.5 43.3 49.1 model |  log
R101 3x 42.8 27.0 46.2 54.9 model |  log

Cityscapes Semantic Segmentation

PointRend + FaPN:

Name lr
sched		 mask
mIoU		 mask
i_IoU		 mask
IoU_sup		 mask
iIoU_sup		 download
R50 1x 80.0 61.3 90.6 78.5 model |  log
R101 1x 80.1 62.2 90.8 78.6 model |  log

ADE20K-150 Semantic Segmentation

MaskFormer + FaPN:

Name mIoU
Single-Scale		 mIoU
Multi-Scale		 download
Swin+Large+IN21K 55.2 56.7 model |  log

COCOStuff-10K Semantic Segmentation

MaskFormer + FaPN:

Name mIoU
Single-Scale		 mIoU
Multi-Scale		 download
R101 39.6 40.6 model |  log

COCO Instance Segmentation

Mask R-CNN + FaPN:

Name lr
sched		 mask
AP		 mask
APs		 box
AP		 box
APs		 download
R50 1x 36.4 18.1 39.8 24.3 model |  log
R101 3x 39.4 20.9 43.8 27.4 model |  log

PointRend + FaPN:

Name lr
sched		 mask
AP		 mask
APs		 box
AP		 box
APs		 download
R50 1x 37.6 18.6 39.4 24.2 model |  log

COCO Panoptic Segmentation

PanopticFPN + FaPN:

Name lr
sched		 PQ mask
mIoU		 St
PQ		 box
AP		 Th
PQ		 download
R50 1x 41.1 43.4 32.5 38.7 46.9 model |  log
R101 3x 44.2 45.7 35.0 43.0 53.3 model |  log

PanopticFCN + FaPN:

Name lr
sched		 PQ mask
mIoU		 St
PQ		 box
AP		 Th
PQ		 download
R50 1x 41.8 42.0 33.1 32.3 47.6 model |  log
R50-600 3x 43.5 43.5 35.1 34.5 49.0 model |  log
You might also like...
EDPN: Enhanced Deep Pyramid Network for Blurry Image Restoration

EDPN: Enhanced Deep Pyramid Network for Blurry Image Restoration Ruikang Xu, Zeyu Xiao, Jie Huang, Yueyi Zhang, Zhiwei Xiong. EDPN: Enhanced Deep Pyra

null 69 Dec 15, 2022
PyTorch implementation for Partially View-aligned Representation Learning with Noise-robust Contrastive Loss (CVPR 2021)
PyTorch implementation for Partially View-aligned Representation Learning with Noise-robust Contrastive Loss (CVPR 2021)

2021-CVPR-MvCLN This repo contains the code and data of the following paper accepted by CVPR 2021 Partially View-aligned Representation Learning with

XLearning Group 33 Nov 1, 2022
(IEEE TIP 2021) Regularized Densely-connected Pyramid Network for Salient Instance Segmentation

RDPNet IEEE TIP 2021: Regularized Densely-connected Pyramid Network for Salient Instance Segmentation PyTorch training and testing code are available.

Yu-Huan Wu 41 Oct 21, 2022
Dense Prediction Transformers

Vision Transformers for Dense Prediction This repository contains code and models for our paper: Vision Transformers for Dense Prediction René Ranftl,

Intel ISL (Intel Intelligent Systems Lab) 1.3k Dec 28, 2022
Implementation of
Implementation of "A MLP-like Architecture for Dense Prediction"

A MLP-like Architecture for Dense Prediction (arXiv) Updates (22/07/2021) Initial release. Model Zoo We provide CycleMLP models pretrained on ImageNet

Shoufa Chen 244 Dec 27, 2022
Dense Prediction Transformers

Vision Transformers for Dense Prediction This repository contains code and models for our paper: Vision Transformers for Dense Prediction René Ranftl,

Intelligent Systems Lab Org 1.3k Jan 2, 2023
This is an official implementation of the High-Resolution Transformer for Dense Prediction.
This is an official implementation of the High-Resolution Transformer for Dense Prediction.

High-Resolution Transformer for Dense Prediction Introduction This is the official implementation of High-Resolution Transformer (HRT). We present a H

HRNet 403 Dec 13, 2022
MPViT:Multi-Path Vision Transformer for Dense Prediction
MPViT:Multi-Path Vision Transformer for Dense Prediction

MPViT : Multi-Path Vision Transformer for Dense Prediction This repository inlcu

Youngwan Lee 272 Dec 20, 2022
Multi-Scale Aligned Distillation for Low-Resolution Detection (CVPR2021)
Multi-Scale Aligned Distillation for Low-Resolution Detection (CVPR2021)

MSAD Multi-Scale Aligned Distillation for Low-Resolution Detection Lu Qi*, Jason Kuen*, Jiuxiang Gu, Zhe Lin, Yi Wang, Yukang Chen, Yanwei Li, Jiaya J

Jia Research Lab 115 Dec 23, 2022
Comments
  • Error when train model.

    Error when train model.

    Hi, I follow your instructions to arrange the project. However, this error still occurs. Could you help me with this issue?

    KeyError: "No object named 'build_resnet_fan_backbone' found in 'BACKBONE' registry!"

    opened by LeoniusChen 2
  • 关于可变形卷积的具体实现

    关于可变形卷积的具体实现

    您好，您的工作是一份非常有价值的工作。由于我在pytorch1.7.1下,编译不了您提供的可变性卷积的库。所以我尝试用mmd实现您的代码，我遇到了一个问题： self.dcpack_L2 = dcn_v2(out_nc, out_nc, 3, stride=1, padding=1, dilation=1, deformable_groups=8, extra_offset_mask=True) 这行代码的mask是根据x算的，还是根据offset算的？您知道具体的计算方法吗？

    opened by xinchenduobian 0
  • Question about the number of

    Question about the number of "out_channels"

    After watching your code, i have a question, about every layer after "FSM" and "FAM" model the output channels are same.Do I understand that right? And i want to know what the number of "out_channels" is set to. Thanks a lot.

    opened by chenhaiwen 1
  • dcn_v2 error RuntimeError: expected scalar type Float but found Half

    dcn_v2 error RuntimeError: expected scalar type Float but found Half

    When running the network, I encountered this problem. Through debugging, I found that the offset in DCN's forword function is a type of float16.So I think this might be the cause of the problem，Do you have a better idea for this problem.

    opened by KingWangJL 8
Owner
Shihua Huang
PhD candidate in PolyU.
Shihua Huang
GitHub http://www.shihuahuang.cn/fapn/
Exploring Relational Context for Multi-Task Dense Prediction [ICCV 2021]

Adaptive Task-Relational Context (ATRC) This repository provides source code for the ICCV 2021 paper Exploring Relational Context for Multi-Task Dense

David Brüggemann 35 Dec 5, 2022
Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision (ICCV 2021)

Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision (ICCV 2021) PyTorch implementation of Learning RAW-to-sRGB Mappings with Inaccurat

Zhilu Zhang 53 Dec 20, 2022
Implementation for our ICCV 2021 paper: Dual-Camera Super-Resolution with Aligned Attention Modules

DCSR: Dual Camera Super-Resolution Implementation for our ICCV 2021 oral paper: Dual-Camera Super-Resolution with Aligned Attention Modules paper | pr

Tengfei Wang 110 Dec 20, 2022
Implementation for our ICCV 2021 paper: Dual-Camera Super-Resolution with Aligned Attention Modules

DCSR: Dual Camera Super-Resolution Implementation for our ICCV 2021 oral paper: Dual-Camera Super-Resolution with Aligned Attention Modules paper | pr

Tengfei Wang 110 Dec 20, 2022
The code repository for "RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection" (ACM MM'21)

RCNet: Reverse Feature Pyramid and Cross-scale Shift Network for Object Detection (ACM MM'21) By Zhuofan Zong, Qianggang Cao, Biao Leng Introduction F

TempleX 9 Jul 30, 2022
Pytorch implementation of Feature Pyramid Network (FPN) for Object Detection

fpn.pytorch Pytorch implementation of Feature Pyramid Network (FPN) for Object Detection Introduction This project inherits the property of our pytorc

Jianwei Yang 912 Dec 21, 2022
Code for the ICCV 2021 Workshop paper: A Unified Efficient Pyramid Transformer for Semantic Segmentation.

Unified-EPT Code for the ICCV 2021 Workshop paper: A Unified Efficient Pyramid Transformer for Semantic Segmentation. Installation Linux, CUDA>=10.0,

null 29 Aug 23, 2022
Tensorflow implementation of the paper "HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences", CVPR 2021.

HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences Tensorflow implementation of the paper "HumanGPS: Geodesic PreServing Feature fo

Google Interns 50 Dec 21, 2022
This is an unofficial implementation of the paper “Student-Teacher Feature Pyramid Matching for Unsupervised Anomaly Detection”.

This is an unofficial implementation of the paper “Student-Teacher Feature Pyramid Matching for Unsupervised Anomaly Detection”.

haifeng xia 32 Oct 26, 2022
An Implementation of SiameseRPN with Feature Pyramid Networks

SiameseRPN with FPN This project is mainly based on HelloRicky123/Siamese-RPN. What I've done is just add a Feature Pyramid Network method to the orig

null 3 Apr 16, 2022