the code of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021)

Overview

RMA-Net

This repo is the implementation of the paper: Recurrent Multi-view Alignment Network for Unsupervised Surface Registration (CVPR 2021).

Paper address: https://arxiv.org/abs/2011.12104

Project webpage: https://wanquanf.github.io/RMA-Net.html

avatar

Prerequisite Installation

The code has been tested with Python3.8, PyTorch 1.6 and Cuda 10.2:

conda create --name rmanet
conda activate rmanet
conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.2 -c pytorch
conda install -c conda-forge igl

Build the cuda extension:

python build_cuda.py

Usage

Pre-trained Models

Download the pre-trained models and put the models in the [YourProjectPath]/pre_trained folder.

Run the registration

To run registration for a single sample, you can run:

python inference.py --weight [pretrained-weight-path] --src [source-obj-path] --tgt [target-obj-path] --iteration [iteration-number] --device_id [gpu-id] --if_nonrigid [1 or 0]

The last argument --if_nonrigid represents if the translation between the source and target is non-rigid (1) or rigid (0). Registration results are listed in the folder named source_deform_results, including the deforming results of different stages. We have given a collection of samples in [YourProjectPath]/samples, and you can run the registration for them by:

sh inference_samples.sh

Datasets

To show how to construct a dataset that can be used in the code, we list some toy pairs in the [YourProjectPath]/toy_dataset folder and give a script to pack them into a bin file:

code for constructing a dataset

Or you can also download the dataset we used in the paper here.

Train & Test

To test on the whole testing set, run:

code for testing on the whole dataset

To train the network, run:

code for training the network

Citation

Please cite this paper with the following bibtex:

@inproceedings{feng2021recurrent,
    author    = {Wanquan Feng and Juyong Zhang and Hongrui Cai and Haofei Xu and Junhui Hou and Hujun Bao},
    title     = {Recurrent Multi-view Alignment Network for Unsupervised Surface Registration},
    booktitle = {{IEEE/CVF} Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2021}
}
Comments
  • Unpickling Error

    Unpickling Error

    Using Python3.8, PyTorch 1.6 and Cuda 10.2, as specified in the repository. While running the sample registration using pretrained weights provided in the link attached to the repository I encountered the following error :

    `Traceback (most recent call last):
      File "inference.py", line 72, in <module>
        rma_net.load_state_dict(torch.load(args.weight),True)
      File "/home/gridraster/anaconda3/envs/rmanet/lib/python3.8/site-packages/torch/serialization.py", line 585, in load
        return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
      File "/home/gridraster/anaconda3/envs/rmanet/lib/python3.8/site-packages/torch/serialization.py", line 755, in _legacy_load
        magic_number = pickle_module.load(f, **pickle_load_args)
    _pickle.UnpicklingError: A load persistent id instruction was encountered,
    but no persistent_load function was specified.
    

    Can you help me fix this issue? What is the pickle version used in the repository while pickling the pretrained weights?

    opened by nandinibajaj 3
  • Deformation problem

    Deformation problem

    Thanks for your excellent work!

    I had some problems when I tried to test RMA-Net on my own data (synthetic data generated from the SMPL model). I have checked my input data (source & target) were stay in a similar coordinate system but different size as your sample data. My result didn't look as good as you show. I have tried to use your sample data, and the results were all perfect. How should I do to get a better result? or if your method has some limitations (maybe it is not suitable for arbitrary body registration)? 1 (The red point cloud is the source, the blue one is the target and the green one is the result from RMA-Net) 2 (The yellow one is my data and another one is your sample data. I am not sure if they are in the same coordinate system but they look similar. )

    opened by daisyranc 1
  • Should I do any preprocess before deformation?

    Should I do any preprocess before deformation?

    Thanks for your excellent work and nice code!

    I have downloaded the code and checkpoint, and tested it one the samples you gave. I got very good results, then I hoped to test my own files.

    I used the SMPL model to generate two poses of the same 'person' (they were set in the same coordinate system), and tested it with the model you trained, but the results were very strange. What prerequisites does this method have for the input data? Should I do any preprocess before deformation?

    Screenshot from 2021-10-13 19-42-57

    opened by crylingya 1
  • Problems downloading the pretrained models

    Problems downloading the pretrained models

    Hi,

    Are there plans to have the pre-trained models available outside Baidu. It looks like currently it requires creating an account on the system in order to download the models. Creating the account though is not straightforward.

    Thank you,

    opened by GMNetto 1
  • Deformation

    Deformation

    When testing on a custom dataset I am getting a deformed result. The source should ideally align itself to the target and take the shape of the target too right?

    My data consists of a simulated point cloud of an object (as source) and a point cloud generated from a real image of the same object (as target). This results in a deformed output that is also not aligned perfectly. Is there any explanation for this? Can you suggest a modification to solve the issue?

    opened by nandinibajaj 2
Owner
Wanquan Feng
Wanquan Feng
Code for the ECCV2020 paper "A Differentiable Recurrent Surface for Asynchronous Event-Based Data"

A Differentiable Recurrent Surface for Asynchronous Event-Based Data Code for the ECCV2020 paper "A Differentiable Recurrent Surface for Asynchronous

Marco Cannici 21 Oct 5, 2022
Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal, multi-exposure and multi-focus image fusion.

U2Fusion Code of U2Fusion: a unified unsupervised image fusion network for multiple image fusion tasks, including multi-modal (VIS-IR, medical), multi

Han Xu 129 Dec 11, 2022
GPU Accelerated Non-rigid ICP for surface registration

GPU Accelerated Non-rigid ICP for surface registration Introduction Preivous Non-rigid ICP algorithm is usually implemented on CPU, and needs to solve

Haozhe Wu 144 Jan 4, 2023
MVP Benchmark for Multi-View Partial Point Cloud Completion and Registration

MVP Benchmark: Multi-View Partial Point Clouds for Completion and Registration [NEWS] 2021-07-12 [NEW ?? ] The submission on Codalab starts! 2021-07-1

PL 93 Dec 21, 2022
Registration Loss Learning for Deep Probabilistic Point Set Registration

RLLReg This repository contains a Pytorch implementation of the point set registration method RLLReg. Details about the method can be found in the 3DV

Felix Järemo Lawin 35 Nov 2, 2022
A multi-scale unsupervised learning for deformable image registration

A multi-scale unsupervised learning for deformable image registration Shuwei Shao, Zhongcai Pei, Weihai Chen, Wentao Zhu, Xingming Wu and Baochang Zha

ShuweiShao 2 Apr 13, 2022
Blender add-on: Add to Cameras menu: View → Camera, View → Add Camera, Camera → View, Previous Camera, Next Camera

Blender add-on: Camera additions In 3D view, it adds these actions to the View|Cameras menu: View → Camera : set the current camera to the 3D view Vie

German Bauer 11 Feb 8, 2022
Implementation of Bidirectional Recurrent Independent Mechanisms (Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neural Networks with Attention over Modules)

BRIMs Bidirectional Recurrent Independent Mechanisms Implementation of the paper Learning to Combine Top-Down and Bottom-Up Signals in Recurrent Neura

Sarthak Mittal 26 May 26, 2022
PyTorch implementation of paper "IBRNet: Learning Multi-View Image-Based Rendering", CVPR 2021.

IBRNet: Learning Multi-View Image-Based Rendering PyTorch implementation of paper "IBRNet: Learning Multi-View Image-Based Rendering", CVPR 2021. IBRN

Google Interns 371 Jan 3, 2023
SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements (CVPR 2021)

SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements (CVPR 2021) This repository contains the official PyTorch implementa

Qianli Ma 133 Jan 5, 2023
[CVPR'21] Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation

Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-view Transformation Weixiang Yang, Qi Li, Wenxi Liu, Yuanlong Yu, Y

null 118 Dec 26, 2022
PyTorch implementation for COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction (CVPR 2021)

Completer: Incomplete Multi-view Clustering via Contrastive Prediction This repo contains the code and data of the following paper accepted by CVPR 20

XLearning Group 72 Dec 7, 2022
ESTDepth: Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks (CVPR 2021)

ESTDepth: Multi-view Depth Estimation using Epipolar Spatio-Temporal Networks (CVPR 2021) Project Page | Video | Paper | Data We present a novel metho

null 65 Nov 28, 2022
Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

null 105 Nov 7, 2022
A Transformer-Based Feature Segmentation and Region Alignment Method For UAV-View Geo-Localization

University1652-Baseline [Paper] [Slide] [Explore Drone-view Data] [Explore Satellite-view Data] [Explore Street-view Data] [Video Sample] [中文介绍] This

Zhedong Zheng 335 Jan 6, 2023
This is the code for CVPR 2021 oral paper: Jigsaw Clustering for Unsupervised Visual Representation Learning

JigsawClustering Jigsaw Clustering for Unsupervised Visual Representation Learning Pengguang Chen, Shu Liu, Jiaya Jia Introduction This project provid

DV Lab 73 Sep 18, 2022
PyTorch implementation of Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network

hierarchical-multi-label-text-classification-pytorch Hierarchical Multi-label Text Classification: An Attention-based Recurrent Network Approach This

Mingu Kang 17 Dec 13, 2022
Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

KaGRMN-DSG_ABSA This repository contains the PyTorch source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated

XingBowen 4 May 20, 2022
UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering

UnsupervisedR&R: Unsupervised Pointcloud Registration via Differentiable Rendering This repository holds all the code and data for our recent work on

Mohamed El Banani 118 Dec 6, 2022