LieTorch: Tangent Space Backpropagation
Introduction
The LieTorch library generalizes PyTorch to 3D transformation groups. Just as torch.Tensor is a multi-dimensional matrix of scalar elements, lietorch.SE3 is a multi-dimensional matrix of SE3 elements. We support common tensor manipulations such as indexing, reshaping, and broadcasting. Group operations can be composed into computation graphs and backpropagation is automatically peformed in the tangent space of each element. For more details, please see our paper:
Tangent Space Backpropagation for 3D Transformation Groups
Zachary Teed and Jia Deng, CVPR 2021
@inproceedings{teed2021tangent,
title={Tangent Space Backpropagation for 3D Transformation Groups},
author={Teed, Zachary and Deng, Jia},
booktitle={Conference on Computer Vision and Pattern Recognition},
year={2021},
}
Installation
Requirements:
- Cuda >= 10.1 (with nvcc compiler)
- PyTorch >= 1.6
We recommend installing within a virtual enviornment. Make sure you clone using the --recursive flag. If you are using Anaconda, the following command can be used to install all dependencies
git clone --recursive https://github.com/princeton-vl/lietorch.git
cd lietorch
conda create -n lie_env
conda activate lie_env
conda install scipy pyyaml pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch
To run the examples, you will need OpenCV and Open3D. Depending on your operating system, OpenCV and Open3D can either be installed with pip or may need to be built from source
pip install opencv-python open3d
Installing:
Clone the repo using the --recursive flag and install using setup.py (may take up to 10 minutes)
git clone --recursive https://github.com/princeton-vl/lietorch.git
python setup.py install
./run_tests.sh
Overview
LieTorch currently supports the 3D transformation groups.
| Group | Dimension | Action |
|---|---|---|
| SO3 | 3 | rotation |
| RxSO3 | 4 | rotation + scaling |
| SE3 | 6 | rotation + translation |
| Sim3 | 7 | rotation + translation + scaling |
Each group supports the following operations:
| Operation | Map | Description |
|---|---|---|
| exp | g -> G | exponential map |
| log | G -> g | logarithm map |
| inv | G -> G | group inverse |
| mul | G x G -> G | group multiplication |
| adj | G x g -> g | adjoint |
| adjT | G x g*-> g* | dual adjoint |
| act | G x R3 -> R3 | action on point (set) |
| act4 | G x P3 -> P3 | action on homogeneous point (set) |
Simple Example:
Compute the angles between all pairs of rotation matrices
import torch
from lietorch import SO3
phi = torch.randn(8000, 3, device='cuda', requires_grad=True)
R = SO3.exp(phi)
# relative rotation matrix, SO3 ^ {100 x 100}
dR = R[:,None].inv() * R[None,:]
# 100x100 matrix of angles
ang = dR.log().norm(dim=-1)
# backpropogation in tangent space
loss = ang.sum()
loss.backward()
Examples
We provide real use cases in the examples directory
- Pose Graph Optimization
- Deep SE3/Sim3 Registrtion
- RGB-D SLAM / VO
Acknowledgements
Many of the Lie Group implementations are adapted from Sophus.
11 Nov 27, 2022
233 Dec 29, 2022
0 Oct 17, 2022
60 Dec 29, 2022
51 Nov 22, 2022
70 Dec 19, 2022
118 Dec 26, 2022
35 Sep 8, 2021
37 Dec 4, 2022
96 Nov 5, 2022
130 Dec 13, 2022
80 Nov 28, 2022
191 Dec 31, 2022
3 Jan 7, 2022
83 Dec 15, 2022
104 Oct 26, 2022
585 Jan 4, 2023
49 Nov 20, 2022
444 Dec 30, 2022