This Repo is the official CUDA implementation of ICCV 2019 Oral paper for CARAFE: Content-Aware ReAssembly of FEatures

Related tags

Deep Learning CARAFE
Overview

Introduction

This Repo is the official CUDA implementation of ICCV 2019 Oral paper for CARAFE: Content-Aware ReAssembly of FEatures.

@inproceedings{Wang_2019_ICCV,
    title = {CARAFE: Content-Aware ReAssembly of FEatures},
    author = {Wang, Jiaqi and Chen, Kai and Xu, Rui and Liu, Ziwei and Loy, Chen Change and Lin, Dahua},
    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
    month = {October},
    year = {2019}
}

Setup CARAFE Operator

There are two ways to setup CARAFE operator.

A. Install mmcv which contains CARAFE.

CARAFE is supported in mmcv. You may install mmcv following the official guideline.

https://github.com/open-mmlab/mmcv

B. Install CARAFE directly from GitHub.

Requirements:

CUDA >= 9.0, Pytorch >= 1.3, Python >= 3.6

Install with pip

pip install git+https://github.com/myownskyW7/CARAFE.git@master

Run gradient check to make sure the operator is successfully compiled

$ python

>>> from carafe import grad_check

C. Compile CARAFE from source.

Requirements:

CUDA >= 9.0, Pytorch >= 1.3, Python >= 3.6

Git clone this repo.

git clone https://github.com/myownskyW7/CARAFE

Setup CARAFE op.

cd CARAFE
python setup.py develop
# or "pip install -v -e ."

Run gradient check to make sure the operator is successfully compiled

$ python

>>> from carafe import grad_check

Usage

import torch
from mmcv.ops.carafe import CARAFEPack
# or "from carafe import CARAFEPack"


x = torch.rand(2, 40, 50, 70)
model = CARAFEPack(channels=40, scale_factor=2)

model = model.cuda()
x = x.cuda()

out = model(x)

print('original shape: ', x.shape)
print('upscaled shape: ', out.shape)

Applications

Projects with CARAFE operators

mmcv

mmdetection

mmediting

Comments
  • something wrong when forward and backpropogation in module

    something wrong when forward and backpropogation in module

    Hi, Thanks you code. I run your code in my module, but I found there was some wrong in computing gradient.[data parallel]

    File "main.py", line 90, in main
    save_model_predictions(p, val_dataloader, model)
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/autograd/grad_mode.py", line 49, in decorate_no_grad
    return func(*args, **kwargs)
    File "/root/paddlejob/workspace/env_run/Multi-Task-Learning-PyTorch-master/evaluation/evaluate_utils.
    py", line 210, in save_model_predictions
    _, _, _, _, output = model(inputs)
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/modules/module.py", line 541, in call
    **result = self.forward(*input, kwargs)
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/parallel/data_parallel.py", line 153, in forward
    return self.gather(outputs, self.output_device)
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/parallel/data_parallel.py", line 165, in gather
    return gather(outputs, output_device, dim=self.dim)
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/parallel/scatter_gather.py", line 68, in gather
    res = gather_map(outputs)
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/parallel/scatter_gather.py", line 63, in gather_map
    return type(out)(map(gather_map, zip(*outputs)))
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/parallel/scatter_gather.py", line 62, in gather_map
    for k in out))
    File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/parallel/scatter_gather.py", line 62, in
    for k in out)) File "/root/paddlejob/workspace/env_run/pytorch_1.2.0a0+8554416-py36/lib/python3.6/site-packages/torc
    h/nn/parallel/scatter_gather.py", line 63, in gather_map
    return type(out)(map(gather_map, zip(*outputs)))
    TypeError: zip argument #1 must support iteration

    If you fix it, I would be really grateful.

    opened by TianhaoFu 0
  • Make CARAFE installable with pip

    Make CARAFE installable with pip

    This PR allows installing CARAFE simply by running:

    pip install git+https://github.com/myownskyW7/CARAFE.git@master

    It also removes the dependency of mmdet in grad_check.py

    opened by maksad 0
  • Not working in half precision

    Not working in half precision

    Installed correctly, grad_check runs without errors as does the sample code. When trying to use either native torch amp or original NVidia one, I receive the same error:

      File "/home/someone/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/someone/anaconda3/lib/python3.7/site-packages/torch/nn/modules/container.py", line 117, in forward
    input = module(input)
  File "/home/someone/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/someone/anaconda3/lib/python3.7/site-packages/carafe/carafe.py", line 250, in forward
    x = self.feature_reassemble(x, mask)
  File "/home/someone/anaconda3/lib/python3.7/site-packages/carafe/carafe.py", line 242, in feature_reassemble
    x = carafe(x, mask, self.up_kernel, self.up_group, self.scale_factor)
  File "/home/someone/anaconda3/lib/python3.7/site-packages/carafe/carafe.py", line 114, in forward
    group_size, scale_factor, routput, output)
RuntimeError: expected scalar type Half but found Float
    opened by tetelias 3
  • CPU implementaiton

    CPU implementaiton

    I have tried CARAFE in my research and was quite happy with the result, now I would like to deploy my model using TorchServe and Amazon Elastic Inference (EI). One of the requirements on EI is "CPU compatibility" and having that variation of CARAFE would greatly help. Is there any timeline about it?

    opened by maksad 0
  • Add CARAFE installation dependency

    Add CARAFE installation dependency

    CARAFE has dependency on torch and when installing with pip one has to make sure that torch is installed first, otherwise the installation raises an error. A much more problematic case is when someone wants to install it with poetry in that case the installation will fail instantly as the setup.py does not indicate installation requirements. This PR solves the problem by simply adding install_requires=['torch'] attribute to the setuptools.setup function.

    Steps to produce

    1. install poetry: pip install poetry
    2. install CARAFE with poetry: poetry add git+https://github.com/myownskyW7/CARAFE.git@master

    Result

      PackageInfoError

  Unable to determine package info for path: /tmp/pypoetry-git-CARAFEvm31k1tj
  
  Fallback egg_info generation failed.
  
  Command ['/tmp/tmp09_lpyl6/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1, and output: 
  Traceback (most recent call last):
    File "setup.py", line 3, in <module>
      from torch.utils.cpp_extension import BuildExtension, CUDAExtension
  ModuleNotFoundError: No module named 'torch'

  at ~/.virtualenvs/parsing/lib/python3.6/site-packages/poetry/inspection/info.py:503 in _pep517_metadata
      499│                     venv.run("python", "setup.py", "egg_info")
      500│                     return cls.from_metadata(path)
      501│                 except EnvCommandError as fbe:
      502│                     raise PackageInfoError(
    → 503│                         path, "Fallback egg_info generation failed.", fbe
      504│                     )
      505│                 finally:
      506│                     os.chdir(cwd.as_posix())

    Expected result:

    Updating dependencies
Resolving dependencies... (1.1s)

Writing lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing carafe (0.0.1 c24cd45)
    opened by maksad 0
  • import error

    import error

    I have installed mmcv-full, but when I try to import CARAFEPack, I encounter the error "ImportError: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by /uac/gds/yangyu/.local/lib/python3.6/site-packages/mmcv/_ext.cpython-36m-x86_64-linux-gnu.so)".

    opened by yuyang16101066 0
Owner
Jiaqi Wang
Jiaqi Wang
GitHub
Official PyTorch implementation of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image", ICCV 2019

PoseNet of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image" Introduction This repo is official Py

Gyeongsik Moon 677 Dec 25, 2022
A Fast and Accurate One-Stage Approach to Visual Grounding, ICCV 2019 (Oral)

One-Stage Visual Grounding ***** New: Our recent work on One-stage VG is available at ReSC.***** A Fast and Accurate One-Stage Approach to Visual Grou

Zhengyuan Yang 118 Dec 5, 2022
[ICCV 2021 Oral] PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers

PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers Created by Xumin Yu*, Yongming Rao*, Ziyi Wang, Zuyan Liu, Jiwen Lu, Jie Zhou

Xumin Yu 317 Dec 26, 2022
An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

PyTorch implementation of SFNet This is the implementation of the paper "SFNet: Learning Object-aware Semantic Correspondence". For more information,

CV Lab @ Yonsei University 87 Dec 30, 2022
The official implementation of ICCV paper "Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds".

Box-Aware Tracker (BAT) Pytorch-Lightning implementation of the Box-Aware Tracker. Box-Aware Feature Enhancement for Single Object Tracking on Point C

Kangel Zenn 5 Mar 26, 2022
[ICCV'21] Official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations

CrowdNav with Social-NCE This is an official implementation for the paper Social NCE: Contrastive Learning of Socially-aware Motion Representations by

VITA lab at EPFL 125 Dec 23, 2022
The official repo of the CVPR2021 oral paper: Representative Batch Normalization with Feature Calibration

Representative Batch Normalization (RBN) with Feature Calibration The official implementation of the CVPR2021 oral paper: Representative Batch Normali

Open source projects of ShangHua-Gao 76 Nov 9, 2022
Official pytorch implementation of "Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization" ACMMM 2021 (Oral)

Feature Stylization and Domain-aware Contrastive Loss for Domain Generalization This is an official implementation of "Feature Stylization and Domain-

null 22 Sep 22, 2022
A PyTorch implementation of SlowFast based on ICCV 2019 paper "SlowFast Networks for Video Recognition"

SlowFast A PyTorch implementation of SlowFast based on ICCV 2019 paper SlowFast Networks for Video Recognition. Requirements Anaconda PyTorch conda in

Hao Ren 8 Dec 23, 2022
Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal Action Localization' (ICCV-21 Oral)

Learning-Action-Completeness-from-Points Official Pytorch Implementation of 'Learning Action Completeness from Points for Weakly-supervised Temporal A

Pilhyeon Lee 67 Jan 3, 2023
Pytorch implementation for "Large-Scale Long-Tailed Recognition in an Open World" (CVPR 2019 ORAL)

Large-Scale Long-Tailed Recognition in an Open World [Project] [Paper] [Blog] Overview Open Long-Tailed Recognition (OLTR) is the author's re-implemen

Zhongqi Miao 761 Dec 26, 2022
Implementation of ICCV2021(Oral) paper - VMNet: Voxel-Mesh Network for Geodesic-aware 3D Semantic Segmentation

VMNet: Voxel-Mesh Network for Geodesic-Aware 3D Semantic Segmentation Created by Zeyu HU Introduction This work is based on our paper VMNet: Voxel-Mes

HU Zeyu 82 Dec 27, 2022
Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

DeepPanoContext (DPC) [Project Page (with interactive results)][Paper] DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context G

Cheng Zhang 66 Nov 16, 2022
Code for "PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation" CVPR 2019 oral

Good news! We release a clean version of PVNet: clean-pvnet, including how to train the PVNet on the custom dataset. Use PVNet with a detector. The tr

ZJU3DV 722 Dec 27, 2022
Based on the paper "Geometry-aware Instance-reweighted Adversarial Training" ICLR 2021 oral

Geometry-aware Instance-reweighted Adversarial Training This repository provides codes for Geometry-aware Instance-reweighted Adversarial Training (ht

Jingfeng 47 Dec 22, 2022
Implementation for paper "STAR: A Structure-aware Lightweight Transformer for Real-time Image Enhancement" (ICCV 2021).

STAR-pytorch Implementation for paper "STAR: A Structure-aware Lightweight Transformer for Real-time Image Enhancement" (ICCV 2021). CVF (pdf) STAR-DC

null 43 Dec 21, 2022
Code for the ICCV 2021 paper "Pixel Difference Networks for Efficient Edge Detection" (Oral).

Pixel Difference Convolution This repository contains the PyTorch implementation for "Pixel Difference Networks for Efficient Edge Detection" by Zhuo

Alex 236 Dec 21, 2022
Official Repo for Ground-aware Monocular 3D Object Detection for Autonomous Driving

Visual 3D Detection Package: This repo aims to provide flexible and reproducible visual 3D detection on KITTI dataset. We expect scripts starting from

Yuxuan Liu 305 Dec 19, 2022
A PyTorch implementation of the baseline method in Panoptic Narrative Grounding (ICCV 2021 Oral)

A PyTorch implementation of the baseline method in Panoptic Narrative Grounding (ICCV 2021 Oral)

Biomedical Computer Vision @ Uniandes 52 Dec 19, 2022