Official implementation of "Articulation Aware Canonical Surface Mapping"

Related tags

Deep Learning acsm
Overview

Articulation-Aware Canonical Surface Mapping

Nilesh Kulkarni, Abhinav Gupta, David F. Fouhey, Shubham Tulsiani

Paper Project Page

Requirements

  • Python 2.7
  • PyTorch tested with 1.2.0 and works with 1.3.0 too.
  • cuda 9.2

For setup and installation refer to docs/install.md instructions.

Setup Evlaution and Training

For ease of acess we provide python scripts that can generate slurm scripts that can be used to generate the results in the paper.

  • Downloading pre-trained model and annotations. Follow setup instructions here

  • Training from scratch. Follow setup instructions here

Citation

If you find the code useful for your research, please consider citing:-

@inproceedings{kulkarni2020articulation,
  title={Articulation-aware Canonical Surface Mapping},
  author={Kulkarni, Nilesh and Gupta, Abhinav and Fouhey, David F and Tulsiani, Shubham},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={452--461},
  year={2020}
}

Future Release

  • Python 3.6
  • PyTorch 1.5
  • PyTorch3D
Comments
  • how do you parameterize part transforms in articulation.

    how do you parameterize part transforms in articulation.

    Thank you for your brilliant work! I have read your excellent paper -ACSM and would like to ask some questions about some details in parameterizing part transformations in articulation procedure.

    As you showed in the supplemental, "We parameterize part transforms as an axis, angle representation." and "Every part’s axis serves as a bias in the network that is learned and is same across the whole category."

    I would like to ask, what is the axis and angle stands for? From my perspective axis = (x0,y0,z0) and angle is represented as quaternion, is it right?

    How to depict the vertices after transformation. p_new = R * (p_old - bias) + bias (1) (R is rotation matrix) Did you do the transformation in this way? So each part of bias is assigned (like pick some fix point) or need to learned by deep network?

    How to learn the angle? Like using ResNet + FC similar to the camera prediction since they are both representation for angle?

    Thank you for checking my problem and I would like to receive your reply.

    opened by anslt 15
  • Inference on other parametric models

    Inference on other parametric models

    Hi, thanks for the great publication. A quick question on using a-csm on other custom parametric models. For inferencing on categories not present in CUBS, is there a way to use the given template models and annotations?

    opened by raincrash 4
  • Is it possible to share the full models for cow and sheep?

    Is it possible to share the full models for cow and sheep?

    Hi,

    Thanks for your excellent work and contribution to the community!

    I noticed the models for cow and sheep are missing (the respective folders in /cachedir/models are empty for these 2 animals). Is it possible to share the content of those folders?

    Thanks a lot in advance!

    Best, Anastasis

    opened by statho 2
  • The PCK under reprojection for horse

    The PCK under reprojection for horse

    Thank you for your excellent work!

    I have run your acsm code and found the PCK under reprojection for horse is much smaller than you show in your paper. The PCK transfer is fine. image as you can see in the following picture, the predict key points are not so close to the ground truth. 0075_img_kp 0075_img_kp_project I have run this to test: python -m acsm.benchmark.pascal.kp_project --name=acsm_horse_8parts --category=horse --parts_file=acsm/part_files/horse.txt --use_html --dl_out_pascal=True --dl_out_imnet=False --split=val --num_train_epoch=400 --num_hypo_cams=8 --env_name=acsm_horse_8parts_pck_val --multiple_cam=True --visuals_freq=5 --visualize=True --n_data_workers=4 --scale_bias=0.75 --resnet_style_decoder=True --resnet_blocks=4 Have I done something wrong?

    opened by linpeisensh 2
  • Why is the part_horse.py with 708 vertices?

    Why is the part_horse.py with 708 vertices?

    As you can see, I have group the obj with Meshmixer and get the same vertices and faces as the original obj. But it can't be grouped. The horse_part.obj you gave me can be grouped and has 708 vertices. So I think it may due to this difference. Could you tell me more about group? image

    Best, Peisen Lin

    opened by linpeisensh 2
  • Bump opencv-python from 3.4.0.12 to 3.4.7.28 in /acsm

    Bump opencv-python from 3.4.0.12 to 3.4.7.28 in /acsm

    Bumps opencv-python from 3.4.0.12 to 3.4.7.28.

    Release notes

    Sourced from opencv-python's releases.

    3.4.7.28

    OpenCV version 3.4.7.

    3.4.6.27

    OpenCV version 3.4.6.

    3.4.5.20

    OpenCV version 3.4.5.

    Once some build issues are solved, next releases will be targeting OpenCV version 4.

    3.4.4.19

    OpenCV version 3.4.4.

    Thanks to Ivan Pozdeev for following fixes and enhancements: #135, #136, #141, #144, #145, #146, #147, #149, #150

    3.4.3.18

    OpenCV version 3.4.3.

    3.4.2.17

    Same as 3.4.2.16 but includes missing x86_64 Linux wheels. Thanks to Krassimir Valev for fixing the build matrix.

    3.4.2.16

    This release bumps OpenCV version to 3.4.2 and adds support for Python 3.7.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Annotations for cow and sheep

    Annotations for cow and sheep

    Hi Nilesh,

    Is it possible to share the annotation and data used for the cow and sheep categories used (just like you did for the horse category)?

    Thanks a lot in advance!

    Best, Anastasis

    opened by statho 1
  • Bump opencv-python from 3.4.0.12 to 4.2.0.32 in /acsm

    Bump opencv-python from 3.4.0.12 to 4.2.0.32 in /acsm

    Bumps opencv-python from 3.4.0.12 to 4.2.0.32.

    Release notes

    Sourced from opencv-python's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    3.4.9.31

    OpenCV version 3.4.9.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    4.1.2.30

    OpenCV version 4.1.2.

    Changes:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Owner
Nilesh Kulkarni
Nilesh Kulkarni
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
The official implementation of NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021]. https://arxiv.org/pdf/2101.12378.pdf

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation [ICLR-2021] Release Notes The offical PyTorch implementation of NeMo, p

Angtian Wang 76 Nov 23, 2022
StyleGAN2-ADA - Official PyTorch implementation

Abstract: Training generative adversarial networks (GAN) using too little data typically leads to discriminator overfitting, causing training to diverge. We propose an adaptive discriminator augmentation mechanism that significantly stabilizes training in limited data regimes.

NVIDIA Research Projects 3.2k Dec 30, 2022
Official implementation of the ICLR 2021 paper

You Only Need Adversarial Supervision for Semantic Image Synthesis Official PyTorch implementation of the ICLR 2021 paper "You Only Need Adversarial S

Bosch Research 272 Dec 28, 2022
Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

This is the official PyTorch implementation of our paper: "Joint Object Detection and Multi-Object Tracking with Graph Neural Networks". Our project website and video demos are here.

Richard Wang 443 Dec 6, 2022
Official implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis https://arxiv.org/abs/2011.13775

CIPS -- Official Pytorch Implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis Requirements pip install -r requi

Multimodal Lab @ Samsung AI Center Moscow 201 Dec 21, 2022
Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

null 364 Dec 14, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022
Official PyTorch Implementation of Unsupervised Learning of Scene Flow Estimation Fusing with Local Rigidity

UnRigidFlow This is the official PyTorch implementation of UnRigidFlow (IJCAI2019). Here are two sample results (~10MB gif for each) of our unsupervis

Liang Liu 28 Nov 16, 2022
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 35 Dec 6, 2022
Official implementation of Self-supervised Graph Attention Networks (SuperGAT), ICLR 2021.

SuperGAT Official implementation of Self-supervised Graph Attention Networks (SuperGAT). This model is presented at How to Find Your Friendly Neighbor

Dongkwan Kim 127 Dec 28, 2022
An official implementation of "SFNet: Learning Object-aware Semantic Correspondence" (CVPR 2019, TPAMI 2020) in PyTorch.

PyTorch implementation of SFNet This is the implementation of the paper "SFNet: Learning Object-aware Semantic Correspondence". For more information,

CV Lab @ Yonsei University 87 Dec 30, 2022
This project is the official implementation of our accepted ICLR 2021 paper BiPointNet: Binary Neural Network for Point Clouds.

BiPointNet: Binary Neural Network for Point Clouds Created by Haotong Qin, Zhongang Cai, Mingyuan Zhang, Yifu Ding, Haiyu Zhao, Shuai Yi, Xianglong Li

Haotong Qin 59 Dec 17, 2022
Official code implementation for "Personalized Federated Learning using Hypernetworks"

Personalized Federated Learning using Hypernetworks This is an official implementation of Personalized Federated Learning using Hypernetworks paper. [

Aviv Shamsian 121 Dec 25, 2022
StyleGAN2 - Official TensorFlow Implementation

StyleGAN2 - Official TensorFlow Implementation

NVIDIA Research Projects 10.1k Dec 28, 2022
Old Photo Restoration (Official PyTorch Implementation)

Bringing Old Photo Back to Life (CVPR 2020 oral)

Microsoft 11.3k Dec 30, 2022
Official implementation of "GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators" (NeurIPS 2020)

GS-WGAN This repository contains the implementation for GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators (NeurIPS

null 46 Nov 9, 2022
Official PyTorch implementation of Spatial Dependency Networks.

Spatial Dependency Networks: Neural Layers for Improved Generative Image Modeling Đorđe Miladinović   Aleksandar Stanić   Stefan Bauer   Jürgen Schmid

Djordje Miladinovic 34 Jan 19, 2022
Official implementation of YOGO for Point-Cloud Processing

You Only Group Once: Efficient Point-Cloud Processing with Token Representation and Relation Inference Module By Chenfeng Xu, Bohan Zhai, Bichen Wu, T

Chenfeng Xu 67 Dec 20, 2022