Back to the Feature with PixLoc
We introduce PixLoc, a neural network for end-to-end learning of camera localization from an image and a 3D model via direct feature alignment. It is presented in our paper:
- Back to the Feature: Learning Robust Camera Localization from Pixels to Pose
- to appear at CVPR 2021
- Authors: Paul-Edouard Sarlin*, Ajaykumar Unagar*, Måns Larsson, Hugo Germain, Carl Toft, Victor Larsson, Marc Pollefeys, Vincent Lepetit, Lars Hammarstrand, Fredrik Kahl, and Torsten Sattler
This repository will host the training and inference code. Please subscribe to this issue if you wish to be notified of the code release.
Abstract
Camera pose estimation in known scenes is a 3D geometry task recently tackled by multiple learning algorithms. Many regress precise geometric quantities, like poses or 3D points, from an input image. This either fails to generalize to new viewpoints or ties the model parameters to a specific scene. In this paper, we go Back to the Feature: we argue that deep networks should focus on learning robust and invariant visual features, while the geometric estimation should be left to principled algorithms. We introduce PixLoc, a scene-agnostic neural network that estimates an accurate 6-DoF pose from an image and a 3D model. Our approach is based on the direct alignment of multiscale deep features, casting camera localization as metric learning. PixLoc learns strong data priors by end-to-end training from pixels to pose and exhibits exceptional generalization to new scenes by separating model parameters and scene geometry. The system can localize in large environments given coarse pose priors but also improve the accuracy of sparse feature matching by jointly refining keypoints and poses with little overhead.
BibTex Citation
Please consider citing our work if you use any of the ideas presented the paper or code from this repo:
@inproceedings{sarlin21pixloc,
author = {Paul-Edouard Sarlin and
Ajaykumar Unagar and
Måns Larsson and
Hugo Germain and
Carl Toft and
Victor Larsson and
Marc Pollefeys and
Vincent Lepetit and
Lars Hammarstrand and
Fredrik Kahl and
Torsten Sattler},
title = {{Back to the Feature}: Learning Robust Camera Localization from Pixels to Pose},
booktitle = {CVPR},
year = {2021},
url = {https://arxiv.org/abs/2103.09213}
}
1 Oct 24, 2021
4 Dec 15, 2022
9 Dec 9, 2022
66 Dec 21, 2022
78 Dec 1, 2022
29 Sep 24, 2022
153 Jan 4, 2023
1.3k Jan 8, 2023
236 Dec 22, 2022
47 Nov 18, 2022
328 Dec 17, 2022
33 Dec 5, 2022
15 Oct 14, 2022
86 Oct 5, 2022
143 Dec 22, 2022
146 Dec 24, 2022
25 Dec 26, 2022
124 Jan 4, 2023
104 Dec 8, 2022