DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction (3DV 2021)

Related tags

Deep Learning DPC
Overview

DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction (3DV 2021)

This repo is the implementation of DPC.

PWC

 

Architecture   Cross Similarity

Tested environment

  • Python 3.6
  • PyTorch 1.6
  • CUDA 10.2

Lower CUDA and PyTorch versions should work as well.

 

Contents

 

Installation

Please follow installation.sh or simply run

bash installation.sh

 

Datasets

The method was evaluated on:

  • SURREAL

    • 230k shapes (DPC uses the first 2k).
    • Dataset website
    • This code downloads and preprocesses SURREAL automatically.

  • SHREC’19

    • 44 Human scans.
    • Dataset website
    • This code downloads and preprocesses SURREAL automatically.

  • SMAL

    • 10000 animal models (2000 models per animal, 5 animals).
    • Dataset website
    • Due to licencing concerns, you should register to SMAL and download the dataset.
    • You should follow data/generate_smal.md after downloading the dataset.

  • TOSCA

    • 41 Animal figures.
    • Dataset website
    • This code downloads and preprocesses TOSCA automatically.

 

Training

For training run

python train_point_corr.py --dataset_name

The code is based on PyTorch-Lightning, all PL hyperparameters are supported. (limit_train/val/test_batches, check_val_every_n_epoch etc.)

 

Tensorboard support

All metrics are being logged automatically and stored in

output/shape_corr/DeepPointCorr/arch_DeepPointCorr/dataset_name_
   
    /run_

Run tesnroboard --logdir= to see the the logs.

Example of tensorboard output:

tensorboard

 

Inference

For testing, simply add --do_train false flag, followed by --resume_from_checkpoint with the relevant checkpoint.

python train_point_corr.py --do_train false  --resume_from_checkpoint

Test phase visualizes each sample, for faster inference pass --show_vis false.

We provide a trained checkpoint repreducing the results provided in the paper, to test and visualize the model run

python train_point_corr.py --show_vis --do_train false --resume_from_checkpoint data/ckpts/surreal_ckpt.ckpt

Results  

Citing & Authors

If you find this repository helpful feel free to cite our publication -

@misc{lang2021dpc,
      title={DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction}, 
      author={Itai Lang and Dvir Ginzburg and Shai Avidan and Dan Raviv},
      year={2021},
      eprint={2110.08636},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Contact: Dvir Ginzburg, Itai Lang

You might also like...
[ICCV 2021 Oral] SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer
[ICCV 2021 Oral] SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer

This repository contains the source code for the paper SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer (ICCV 2021 Oral). The project page is here.

AllenXiang 65 Dec 26, 2022
NeRF visualization library under construction

NeRF visualization library using PlenOctrees, under construction pip install nerfvis Docs will be at: https://nerfvis.readthedocs.org import nerfvis s

Alex Yu 196 Jan 4, 2023
A knowledge base construction engine for richly formatted data
A knowledge base construction engine for richly formatted data

Fonduer is a Python package and framework for building knowledge base construction (KBC) applications from richly formatted data. Note that Fonduer is

HazyResearch 386 Dec 5, 2022
Detectron2-FC a fast construction platform of neural network algorithm based on detectron2

What is Detectron2-FC Detectron2-FC a fast construction platform of neural network algorithm based on detectron2. We have been working hard in two dir

董晋宗 9 Jun 6, 2022
CVPR 2021 Official Pytorch Code for UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

UC2 UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu,

Mingyang Zhou 28 Dec 30, 2022
Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds This is the official code implementation for the paper "Spatio-temporal Se

Hesper 63 Jan 5, 2023
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

The Official PyTorch Implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Shiyi Lan 3 Oct 15, 2021
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.
The PyTorch implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision.

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision The PyTorch implementation of DiscoBox: Weakly Supe

Shiyi Lan 1 Oct 23, 2021
SurfEmb (CVPR 2022) - SurfEmb: Dense and Continuous Correspondence Distributions

SurfEmb SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings Rasmus Laurvig Haugard, A

Rasmus Haugaard 56 Nov 19, 2022
Comments
  • Question about using chamfer as self-construction loss.

    Question about using chamfer as self-construction loss.

    Hi! Thanks for releasing the code of your wonderful work.

    For self-construction loss, we are supposed to know the ground-truth point. Then why not just use a supervised loss using the gt point, but a unsupervised chamfer distance. Is there any specific consideration?

    opened by GostInShell 4
  • Evalutating on a larger point cloud

    Evalutating on a larger point cloud

    Hello,

    Congrats on the great work! Can you please suggest what if I'd want to run this method on a dataset with potentially 5k / 8k / 200k vertices? For example, if I'd want to compare your method on the FAUST or SHREC'19 original dataset, how should I proceed?

    Thanks!

    opened by Sentient07 3
  • I couldn‘t repreduce the result

    I couldn‘t repreduce the result

    Hello，I try to use the checkpoints you provided to repreducing the results provided in the paper, but I can't not repreduce the result。 Here is the command I wrote: python train_point_corr.py --do_train False --gpus 0 --num_data_workers 8 --resume_from_checkpoint ./data/ckpts/shrec_ckpt.ckpt --default_root_dir ./output/

    According to the paper, I should get 15.3% acc at 1% tolerance. But I only get 14.3%.

    Could you tell me why? Thanks

    opened by black-prince222 2
  • Missing File:

    Missing File: "./data/surreal_template.ply"

    Hi,

    First of all, thanks for your great work. It's a really interesting one to work upon. While I studied the code of this work, I found that there is a file missing if I tried to train the DPC from scratch on the Surreal dataset, referring to this line of code. The 'surreal_template.ply' cannot be found in the data directory. I tried to look up the repository of 3D-CODED, but I did not find the 'surreal_template.ply' file. If you are pleased, could you provide surreal_template.ply in this repository?

    opened by PinkViolet 1
Owner
Dvir Ginzburg
Computer vision researcher. Currently pursuing my Ph.D. at Tel-Aviv University on deep neural networks for point clouds.
Dvir Ginzburg
GitHub
PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Reference-Based-Sketch-Image-Colorization-ImageNet This is a PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization usin

Yuzhi ZHAO 11 Jul 28, 2022
RGBD-Net - This repository contains a pytorch lightning implementation for the 3DV 2021 RGBD-Net paper.

[3DV 2021] We propose a new cascaded architecture for novel view synthesis, called RGBD-Net, which consists of two core components: a hierarchical depth regression network and a depth-aware generator network.

Phong Nguyen Ha 4 May 26, 2022
Self-Learned Video Rain Streak Removal: When Cyclic Consistency Meets Temporal Correspondence

In this paper, we address the problem of rain streaks removal in video by developing a self-learned rain streak removal method, which does not require any clean groundtruth images in the training process.

Yang Wenhan 44 Dec 6, 2022
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution

FAU Implementation of the paper: Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution. Yingruo

Evelyn 78 Nov 29, 2022
This repo is a PyTorch implementation for Paper "Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds"

Unsupervised Learning for Cuboid Shape Abstraction via Joint Segmentation from Point Clouds This repository is a PyTorch implementation for paper: Uns

Kaizhi Yang 42 Dec 9, 2022
[3DV 2020] PeeledHuman: Robust Shape Representation for Textured 3D Human Body Reconstruction

PeeledHuman: Robust Shape Representation for Textured 3D Human Body Reconstruction International Conference on 3D Vision, 2020 Sai Sagar Jinka1, Rohan

Rohan Chacko 39 Oct 12, 2022
Shape Matching of Real 3D Object Data to Synthetic 3D CADs (3DV project @ ETHZ)

Real2CAD-3DV Shape Matching of Real 3D Object Data to Synthetic 3D CADs (3DV project @ ETHZ) Group Member: Yue Pan, Yuanwen Yue, Bingxin Ke, Yujie He

null 24 Jun 22, 2022
Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021)

Style-based Point Generator with Adversarial Rendering for Point Cloud Completion (CVPR 2021) An efficient PyTorch library for Point Cloud Completion.

Microsoft 119 Jan 2, 2023
Code for "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds", CVPR 2021

PV-RAFT This repository contains the PyTorch implementation for paper "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clou

Yi Wei 43 Dec 5, 2022