A robust pointcloud registration pipeline based on correlation.

Related tags

Deep Learning phaser
Overview

PHASER: A Robust and Correspondence-Free Global Pointcloud Registration

Ubuntu 18.04+ROS Melodic: Build Status


Overview

Pointcloud registration using correspondences is inefficient and prone to errors in the many steps of correspondence extraction, description, and matching. Similarly, the most widespread registration methods work only locally, requiring an initial guess already close to the true solution, something unaffordable in real robotic deployments. We propose an algorithm for the registration of partially overlapping pointclouds that operates at the global level and on the raw data, i.e., no initial guess as well as no candidate matches are required. We exploit the properties of Fourier analysis to derive a novel registration pipeline based on the cross-correlation of the phases.

Packages

PHASER is composed of the following packages:

  • phaser_core: The registration core of PHASER. Contains the spherical and spatial correlation.
  • phaser_ros: This is a ROS wrapper to use the PHASER as a registration framework. Hardly used anymore.
  • phaser_common: Exposes common classes, utils and models.
  • phaser_pre: Experimental preprocessing of pointcloud data.
  • phaser_viz: Provides visualization functions.
  • phaser_test_data: Contains example data as PLYs.
  • phaser_share: Provides run and build scripts.

Installation

PHASER requires ROS and some other dependencies to be installed:

Dependencies

  # Some standard requirements
  sudo apt-get install -y doxygen autotools-dev \
     dh-autoreconf libboost-all-dev python-setuptools git g++ cppcheck \
     libgtest-dev python-git pylint \
     python-termcolor liblog4cplus-dev cimg-dev python-wstool \
     python-catkin-tools \

   # Ubuntu 18.04 / ROS Melodic.
   sudo apt-get install -y clang-format-6.0 ros-melodic-pcl-conversions \
     libpcl-dev libnlopt-dev \

Important: Currently, PHASER also requires nvcc for compilation as most-recent experiments deal with performing the FFTs on the GPU.

For the remaining package dependencies, run within the caktin workspace

  wstool init
  wstool merge phaser/dependencies.rosinstall
  wstool update

Building the project:

  catkin build phaser_ros

Optionally one can build an run all unit tests using:

  ./phaser_share/run_build_tests

However, this might take some minutes to finish.

Example

The package phaser_core provides a simple test driver to run PHASER using two pointclouds stored as .ply files. Additionally, run script for the test driver is provided in the phaser_share directory.

The initial alignment of the two pointclouds is as follows: PHASER Input Example

By running

./phaser_share/run_phaser_core_driver

the registered pointcloud is written to disk as registered.ply. You might need to adapt the source and target pointcloud paths. Furthermore, other pointcloud examples can be found in the phaser_test_data/test_clouds/os0/ directory.

In this particular case, the registration is configured to be very fine. Thus, it will take a few seconds to finish: PHASER Registered Example

Development Guidelines

Reference

Our paper is available at
Bernreiter, Lukas, Lionel Ott, Juan Nieto, Roland Siegwart, and Cesar Cadena. "PHASER: A Robust and Correspondence-Free Global Pointcloud Registration." IEEE Robotics and Automation Letters 6, no. 2 (2021): 855-862. [Link] [ArXiv].

BibTex:

@article{bernreiter2021phaser,
  title={PHASER: A Robust and Correspondence-Free Global Pointcloud Registration},
  author={Bernreiter, Lukas and Ott, Lionel and Nieto, Juan and Siegwart, Roland and Cadena, Cesar},
  journal={IEEE Robotics and Automation Letters},
  volume={6},
  number={2},
  pages={855--862},
  year={2021},
  publisher={IEEE}
}
Comments
  • Add depdendencies

    Add depdendencies

    General

    This PR adds the dependencies as either rosinstall or incorporated submodule. Right now it is still pulling maplab_dependencies. This however, changes in the future once it is clear which deps are required.

    Changelog

    • Changes the overall structure of the project to facilitate submodules as dependencies.
    opened by LBern 27
  • Prepare Evaluatiion.

    Prepare Evaluatiion.

    General

    This PR generally prepares the experiments for the evaluation and comparison to the sota.

    Current ToDo's:

    • [x] Add ICP
    • [x] Add NDT
    • [x] Add CPD
    • [x] Add G-ICP

    Changelog

    opened by LBern 12
  • Catkin install

    Catkin install

    Hi, interested in using this code however it seems like there is a fair amount of configuration required. A dockerfile would be handy (e.g. it would help with open issues https://github.com/ethz-asl/phaser/issues/62 and https://github.com/ethz-asl/phaser/issues/61). Happy to PR it but can't get this working.

    The catkin build phaser_ros line errors with #15 0.593 [build] Error: Unable to find source space '/phaser/src'.

    Not familiar with catkin. What am I missing here?

    Dockerfile I'm using:

    FROM ubuntu:18.04
    
    ENV DEBIAN_FRONTEND noninteractive
    
    RUN apt update
    RUN apt install -y doxygen autotools-dev \
        dh-autoreconf libboost-all-dev python-setuptools git g++ cppcheck \
        libgtest-dev python-git pylint \
        python-termcolor liblog4cplus-dev cimg-dev python-wstool python-pip \
        clang-format-6.0 curl \
        libpcl-dev libnlopt-dev
    
    RUN echo "deb http://packages.ros.org/ros/ubuntu bionic main" | tee /etc/apt/sources.list.d/ros-latest.list
    RUN curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | apt-key add -
    RUN apt update
    RUN apt install -y ros-melodic-pcl-conversions
    
    RUN pip install -U catkin_tools
    
    RUN git clone https://github.com/jalexvig/phaser.git
    
    WORKDIR phaser
    
    RUN wstool init && wstool merge dependencies.rosinstall && wstool update
    
    RUN catkin build phaser_ros
    
    RUN apt install -y zsh vim
    RUN wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | zsh || true
    
    CMD [ "zsh" ]
    
    opened by jalexvig 10
  • Add a tool for the spherical feature conversion

    Add a tool for the spherical feature conversion

    General

    This PR adds a new tool that reads an exported feature folder and performs the spherical Fourier transform. The output will be the magnitude of the signals.

    opened by LBern 10
  • Add Deployment Scripts

    Add Deployment Scripts

    General

    This PR introduces scripts to create a docker container for PHASER. Basically provides more detailed information on how to build the repo (#64)

    How to use it

    Install docker using

    sudo apt install docker.io docker
    

    then run the script

    ./phaser_deploy/build_docker.sh
    

    This creates a new ubuntu:18.04 container, installs all the dependencies, and builds PHASER.

    PHASER can then be run using

    docker run -it phaser.core (bash/rosrun/....) 
    
    opened by LBern 7
  • Add a directional distribution for the orientation uncertainty

    Add a directional distribution for the orientation uncertainty

    General

    After the correlation the resulting ZYZ angles are converted to get the XYZ angles for the orientation. This is done by converting them to quaternions which can be distributed according a Bingham or vMF distribution.

    Changelog

    • Adds a Bingham distribution and corresponding gtests to phaser_common
    • Adds a Bingham Mixture distribution
    • Adds new peak-based eval methods
    opened by LBern 7
  • Full Registration Algorithm

    Full Registration Algorithm

    General

    This PR is to keep track of the current issues and fix to do.

    • [x] Rotational alignment
    • [x] Translation alignment
    • [x] Uncertainty analysis
    • [x] Unit tests for the components
    • [x] Full pipeline combining the uncertainty analysis with the registration
    opened by LBern 7
  • Add Laplacian Pyramid Fusion

    Add Laplacian Pyramid Fusion

    General

    This PR adds a new fusion component, namely the Laplacian pyramid. Given an input spectrum, we first derive multiple low-pass pyramid levels. Consequently, the difference between the previous level and the current low-pass pyramid results in the Laplacian pyramid.

    opened by LBern 6
  • Implement a spatial zero padding

    Implement a spatial zero padding

    General

    We apply an averaging of the voxels for the sampling. To compensate for the introduced partial voluming effect, we perform zero-padding of the signal.

    opened by LBern 4
  • Add a low-pass for the translational estimation

    Add a low-pass for the translational estimation

    General

    This PR generally adds the possibility to filter out all frequencies except for the lower ones. This is based on the idea that lower frequencies contain the major amount of signal energies and are less affected by noise.

    opened by LBern 4
  • Change from pcl_catkin to system pcl

    Change from pcl_catkin to system pcl

    General

    For easier integration in order frameworks, pcl_catkin pulls too many things into the workspace. Consequently, switching to system pcl makes the integration easier.

    Changelog

    • Removed the debug-visualizer since it is not supported by system pcl
    opened by LBern 4
Owner
ETHZ ASL
ETHZ ASL
Scikit-event-correlation - Event Correlation and Forecasting over High Dimensional Streaming Sensor Data algorithms

scikit-event-correlation Event Correlation and Changing Detection Algorithm Theo

Intellia ICT 5 Oct 30, 2022
PointCloud Annotation Tools, support to label object bound box, ground, lane and kerb

PointCloud Annotation Tools, support to label object bound box, ground, lane and kerb

halo 368 Dec 6, 2022
Registration Loss Learning for Deep Probabilistic Point Set Registration

RLLReg This repository contains a Pytorch implementation of the point set registration method RLLReg. Details about the method can be found in the 3DV

Felix JƤremo Lawin 35 Nov 2, 2022
[PyTorch] Official implementation of CVPR2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency". https://arxiv.org/abs/2103.05465

PointDSC repository PyTorch implementation of PointDSC for CVPR'2021 paper "PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency",

null 153 Dec 14, 2022
PyTorch implementation of NeurIPS 2021 paper: "CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration"

PyTorch implementation of NeurIPS 2021 paper: "CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration"

null 76 Jan 3, 2023
GeoTransformer - Geometric Transformer for Fast and Robust Point Cloud Registration

Geometric Transformer for Fast and Robust Point Cloud Registration PyTorch imple

Zheng Qin 220 Jan 5, 2023
DWIPrep is a robust and easy-to-use pipeline for preprocessing of diverse dMRI data.

DWIPrep: A Robust Preprocessing Pipeline for dMRI Data DWIPrep is a robust and easy-to-use pipeline for preprocessing of diverse dMRI data. The transp

Gal Ben-Zvi 1 Jan 9, 2023
[TIP 2020] Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion

Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion Code for Multi-Temporal Scene Classification and Scene Ch

Lixiang Ru 33 Dec 12, 2022
LBK 35 Dec 26, 2022
Code for "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds", CVPR 2021

PV-RAFT This repository contains the PyTorch implementation for paper "PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clou

Yi Wei 43 Dec 5, 2022
šŸ§  A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation.', ECCV 2016

Deep CORAL A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation. B Sun, K Saenko, ECCV 2016' Deep CORAL can learn

Andy Hsu 200 Dec 25, 2022
Extreme Rotation Estimation using Dense Correlation Volumes

Extreme Rotation Estimation using Dense Correlation Volumes This repository contains a PyTorch implementation of the paper: Extreme Rotation Estimatio

Ruojin Cai 29 Nov 18, 2022
Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting

Autoformer (NeurIPS 2021) Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting Time series forecasting is a c

THUML @ Tsinghua University 847 Jan 8, 2023
An official source code for paper Deep Graph Clustering via Dual Correlation Reduction, accepted by AAAI 2022

Dual Correlation Reduction Network An official source code for paper Deep Graph Clustering via Dual Correlation Reduction, accepted by AAAI 2022. Any

yueliu1999 109 Dec 23, 2022
An ML & Correlation platform for transforming disparate data points of interest into usable intelligence.

SSIDprobeCollector An ML & Correlation platform for transforming disparate data points of interest into usable intelligence. At a High level the platf

Bill Reyor 1 Jan 30, 2022
Code for One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning (AAAI 2022)

One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning (AAAI 2022) Paper | Demo Requirements Python >= 3.6 , Pytorch >

FuxiVirtualHuman 84 Jan 3, 2023
Exploring the Dual-task Correlation for Pose Guided Person Image Generation

Dual-task Pose Transformer Network The source code for our paper "Exploring Dual-task Correlation for Pose Guided Person Image Generationā€œ (CVPR2022)

null 63 Dec 15, 2022
Framework for joint representation learning, evaluation through multimodal registration and comparison with image translation based approaches

CoMIR: Contrastive Multimodal Image Representation for Registration Framework ?? Registration of images in different modalities with Deep Learning ??

Methods for Image Data Analysis - MIDA 55 Dec 9, 2022
An algorithm that handles large-scale aerial photo co-registration, based on SURF, RANSAC and PyTorch autograd.

An algorithm that handles large-scale aerial photo co-registration, based on SURF, RANSAC and PyTorch autograd.

Luna Yue Huang 41 Oct 29, 2022