UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering

Related tags

Deep Learning point-cloud registration alignment 3d-vision differentiable-rendering
Overview

UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering

This repository holds all the code and data for our recent work on unsupervised point cloud registration:

UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering
Mohamed El Banani, Luya Gao, Justin Johnson

If you find this code useful, please consider citing:

@inProceedings{elbanani2021unsupervisedrr,
  title={UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering},
  author={El Banani, Mohamed and Gao, Luya and Johnson, Justin},
  year={2021},
}

If you have any questions about the paper or the code, please feel free to email me at [email protected]

Usage Instructions

  1. How to setup your environment?
  2. How to download and setup the datasets?
  3. How to train models?
  4. How to run inference with pretrained checkpoints?

Acknowledgments

We would like to thank the reviewers and area chairs for their valuable comments and suggestions. We also thank Nilesh Kulkarni, Karan Desai, Richard Higgins, and Max Smith for many helpful discussions and feedback on early drafts of this work.

We would also like to acknowledge the following repositories and users for making great code openly available for us to use:

Comments
  • The consumption of ScanNet RGB-D vedio dataset

    The consumption of ScanNet RGB-D vedio dataset

    Thanks for your great work. Would you like to share the final storge occupancy of ScanNet RGB-D video dataset? I first download the .scens of scene0000_00 and utilize the reader function provided by ScanNet, and I found the storge occupancy of this scene is over 5G. The number of training scenes is over 1000, so it will lead to an expansive storage cost. Limited by my hardware conditions, I need to know the final storage occupancy of the processed dataset.

    And the time cost of processing seems to be a lot? How much time the whole processing will be consumed?

    Will this project use all scenes of ScanNet as training examples? Or take only 3 scenes as training examples listed in the datasets.md in docs?

    Thanks!

    opened by lzhnb 9
  • Whether the calculated Rt is differentiable

    Whether the calculated Rt is differentiable

    Hello, I observed on your code that when Rt is calculated according to the point correspondence, it is solved by SVD. But is the loss obtained in this way derivable? image

    https://github.com/mbanani/unsupervisedRR/blob/4ba5b1cc76080f5c6fd99cec155453f43b547bcd/unsupervisedRR/models/alignment.py#L127

    opened by linxcan 2
  • Could you please give me training instructions

    Could you please give me training instructions

    Hello！Thank you for your work. I am very interested in your work. Could you please give me training instructions? I want to try to train. Thank you very much.

    opened by linxcan 2
  • Shared layer intentional ?

    Shared layer intentional ?

    The resnet-18 layer that is used by the encoder do not seem to be deepcopied anywhere. Which means that layer1 and layer1 do share weights.

    Since there is no mention of it in the paper, one might wonder if this is intentional, or potentially a bug.

    opened by gleize 2
  • About scannet

    About scannet

    Thank you for your sharing codes. I have downloaded scannet, but found that its intrinsic include intrinsic_depth and intrinsic_color. However, I find the intrinsic for depth is replaced with the intrinsic for color in code 'make_scannet_dict' . Is it okay to do this?

    opened by phdymz 2
  • Config File For reproducing experiments

    Config File For reproducing experiments

    Hi,

    I could not locate the configuration files used for the training part. I want to train the network on scannet dataset. Can you please point to where I can find config files.

    Thanks and regards, Sarvesh

    opened by ThakurSarveshGit 1
  • Small fixes/improvements

    Small fixes/improvements

    Hey, Thanks again for the code and the checkpoint! Here are some small changes that could help others

    • Some typos and fixing the link to https://mbanani.github.io/unsupervisedrr/
    • changing pip command to python -m pip inside docs/environment.md to make sure that the pip inside conda is used
    opened by Sebastian-Jung 1
Owner
Mohamed El Banani
Ph.D. Student in Computer Science
Mohamed El Banani
GitHub https://mbanani.github.io/unsupervisedrr
PointCloud Annotation Tools, support to label object bound box, ground, lane and kerb

PointCloud Annotation Tools, support to label object bound box, ground, lane and kerb

halo 368 Dec 6, 2022
Registration Loss Learning for Deep Probabilistic Point Set Registration

RLLReg This repository contains a Pytorch implementation of the point set registration method RLLReg. Details about the method can be found in the 3DV

Felix Järemo Lawin 35 Nov 2, 2022
This repository contains the code for the CVPR 2020 paper "Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision"

Differentiable Volumetric Rendering Paper | Supplementary | Spotlight Video | Blog Entry | Presentation | Interactive Slides | Project Page This repos

null 697 Jan 6, 2023
Official Repo for ICCV2021 Paper: Learning to Regress Bodies from Images using Differentiable Semantic Rendering

[ICCV2021] Learning to Regress Bodies from Images using Differentiable Semantic Rendering Getting Started DSR has been implemented and tested on Ubunt

Sai Kumar Dwivedi 83 Nov 27, 2022
Differentiable Neural Computers, Sparse Access Memory and Sparse Differentiable Neural Computers, for Pytorch

Differentiable Neural Computers and family, for Pytorch Includes: Differentiable Neural Computers (DNC) Sparse Access Memory (SAM) Sparse Differentiab

ixaxaar 302 Dec 14, 2022
the code of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021)

RMA-Net This repo is the implementation of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021). Paper

Wanquan Feng 205 Nov 9, 2022
A multi-scale unsupervised learning for deformable image registration

A multi-scale unsupervised learning for deformable image registration Shuwei Shao, Zhongcai Pei, Weihai Chen, Wentao Zhu, Xingming Wu and Baochang Zha

ShuweiShao 2 Apr 13, 2022
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

Ren Yurui 261 Jan 9, 2023
The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic Neural Rendering"

Website | ArXiv | Get Start | Video PIRenderer The source code of the ICCV2021 paper "PIRenderer: Controllable Portrait Image Generation via Semantic

Ren Yurui 81 Sep 25, 2021
Python implementation of Wu et al (2018)'s registration fusion

reg-fusion Projection of a central sulcus probability map using the RF-ANTs approach (right hemisphere shown). This is a Python implementation of Wu e

Dan Gale 26 Nov 12, 2021
[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

PointDSC repository PyTorch implementation of PointDSC for CVPR'2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency",

null 153 Dec 14, 2022
Python Implementation of the CoronaWarnApp (CWA) Event Registration

Python implementation of the Corona-Warn-App (CWA) Event Registration This is an implementation of the Protocol used to generate event and location QR

MaZderMind 17 Oct 5, 2022
Vision Transformer for 3D medical image registration (Pytorch).

ViT-V-Net: Vision Transformer for Volumetric Medical Image Registration keywords: vision transformer, convolutional neural networks, image registratio

Junyu Chen 192 Dec 20, 2022
Framework for joint representation learning, evaluation through multimodal registration and comparison with image translation based approaches

CoMIR: Contrastive Multimodal Image Representation for Registration Framework ?? Registration of images in different modalities with Deep Learning ??

Methods for Image Data Analysis - MIDA 55 Dec 9, 2022
[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration This repository contains the implementation of our paper Locally Aware Pi

sfwang 70 Dec 19, 2022
An algorithm that handles large-scale aerial photo co-registration, based on SURF, RANSAC and PyTorch autograd.

An algorithm that handles large-scale aerial photo co-registration, based on SURF, RANSAC and PyTorch autograd.

Luna Yue Huang 41 Oct 29, 2022
MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration

MVP Benchmark: Multi-View Partial Point Clouds for Completion and Registration [NEWS] 2021-07-12 [NEW ?? ] The submission on Codalab starts! 2021-07-1

PL 93 Dec 21, 2022
Compute descriptors for 3D point cloud registration using a multi scale sparse voxel architecture

MS-SVConv : 3D Point Cloud Registration with Multi-Scale Architecture and Self-supervised Fine-tuning Compute features for 3D point cloud registration

null 42 Jul 25, 2022