template-pose
Pytorch implementation of "Templates for 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions" paper (accepted to CVPR 2022)
Van Nguyen Nguyen, Yinlin Hu, Yang Xiao, Mathieu Salzmann and Vincent Lepetit
Check out our paper and webpage for details!
If our project is helpful for your research, please consider citing :
@inproceedings{nguyen2022template,
title={Templates for 3D Object Pose Estimation Revisited: Generalization to New objects and Robustness to Occlusions},
author={Nguyen, Van Nguyen and Hu, Yinlin and Xiao, Yang and Salzmann, Mathieu and Lepetit, Vincent},
booktitle={Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
year={2022}}
Table of Content
Methodology
🧑🎓
We introduce template-pose, which estimates 3D pose of new objects (can be very different from the training ones, i.e LINEMOD dataset) with only their 3D models. Our method requires neither a training phase on these objects nor images depicting them.
Two settings are considered in this work:
| Dataset | Predict ID object | In-plane rotation |
|---|---|---|
| (Occlusion-)LINEMOD | Yes | No |
| T-LESS | No | Yes |
Installation
👨🔧
We recommend creating a new Anaconda environment to use template-pose. Use the following commands to setup a new environment:
conda env create -f environment.yml
conda activate template
Optional: Installation of BlenderProc is required to render synthetic images. It can be ignored if you use our provided template. More details can be found in Datasets.
Datasets
😺
🔌
Before downloading the datasets, you may change this line to define the $ROOT folder (to store data and results).
There are two options:
- To download our pre-processed datasets (15GB) + SUN397 dataset (37GB)
./data/download_preprocessed_data.sh
Optional: You can download with following gdrive links and unzip them manually. We recommend keeping $DATA folder structure as detailed in ./data/README to keep pipeline simple:
- LINEMOD and Occlusion-LINEMOD (3GB) -> then run
python -m data.crop_image_linemod - T-LESS (11GB)
- Templates (both T-LESS and LINEMOD) (1.7GB)
- Dataframes including query-template pairwise used for training (11MB)
- SUN397, randomized background for training on T-LESS (37GB)
- To download the original datasets and process them from scratch (process GT poses, render templates, compute nearest neighbors). All the main steps are detailed in ./data/README.
./data/download_and_process_from_scratch.sh
For any training with backbone ResNet50, we initialise with pretrained features of MOCOv2 which can be downloaded with the following command:
python -m lib.download_weight --model_name MoCov2
T-LESS
🔌
1. To launch a training on T-LESS:
python train_tless.py --config_path ./config_run/TLESS.json
2. To reproduce the results on T-LESS:
To download pretrained weights (by default, they are saved at $ROOT/pretrained/TLESS.pth):
python -m lib.download_weight --model_name TLESS
Optional: You can download manually with this link
To evaluate model with the pretrained weight:
python test_tless.py --config_path ./config_run/TLESS.json --checkpoint $ROOT/pretrained/TLESS.pth
LINEMOD and Occlusion-LINEMOD
😺
1. To launch a training on LINEMOD:
python train_linemod.py --config_path config_run/LM_$backbone_$split_name.json
For example, with “base" backbone and split #1:
python train_linemod.py --config_path config_run/LM_baseNetwork_split1.json
2. To reproduce the results on LINEMOD:
To download pretrained weights (by default, they are saved at $ROOT/pretrained):
python -m lib.download_weight --model_name LM_$backbone_$split_name
Optional: You can download manually with this link
To evaluate model with a checkpoint_path:
python test_linemod.py --config_path config_run/LM_$backbone_$split_name.json --checkpoint checkpoint_path
For example, with “base" backbone and split #1:
python -m lib.download_weight --model_name LM_baseNetwork_split1
python test_linemod.py --config_path config_run/LM_baseNetwork_split1.json --checkpoint $ROOT/pretrained/LM_baseNetwork_split1.pth
Acknowledgement
The code is adapted from PoseContrast, DTI-Clustering, CosyPose and BOP Toolkit. Many thanks to them!
The authors thank Martin Sundermeyer, Paul Wohlhart and Shreyas Hampali for their fast reply, feedback!
Contact
If you have any question, feel free to create an issue or contact the first author at [email protected]
3 Aug 20, 2022
141 Dec 30, 2022
183 Jan 3, 2023
23 Nov 29, 2022
37 Nov 27, 2022
35 Nov 22, 2022
24 Nov 1, 2022
57 Dec 27, 2022
150 Dec 26, 2022
113 Dec 21, 2022
31 Sep 27, 2022
50 Dec 17, 2022
151 Dec 30, 2022
294 Jan 1, 2023
21 Dec 22, 2022
47 Dec 4, 2022
64 Jan 7, 2023
1.4k Jan 1, 2023
73 Dec 18, 2022