Implementation of paper "DeepTag: A General Framework for Fiducial Marker Design and Detection"

Related tags

Deep Learning deeptag-pytorch
Overview

Implementation of paper DeepTag: A General Framework for Fiducial Marker Design and Detection.

Project page: https://herohuyongtao.github.io/research/publications/deep-tag/.

Overview

DeepTag is a general framework for fiducial marker design and detection, which supports existing and newly-designed marker families. DeepTag is a two-stage marker detection pipeline:

  • Stage-1: detect ROIs of potential markers;
  • Stage-2: detect keypoints and digital symbols inside each ROI, then determine 6-DoF pose and marker ID.

pipeline

How to run

  • For image input: 
    python test_deeptag.py --config config_image.json
  • For video input: 
    python test_deeptag.py --config config_video.json

The configuration file is in JSON format. Please modify the configurations to fit your needs. Example configurations files for image and video input are provided (i.e., config_image.json and config_video.json).

Detail explaination of configuration file:

  • is_video: {0, 1} for image/video respectively.
  • filepath: path of input image/video (use 0 for webcam input).
  • family: marker family, currently support {apriltag, aruco, artoolkitplus, runetag, topotag, apriltagxo}.
  • hamming_dist: Hamming dist for checking the marker library; normally, 4 works well enough.
  • codebook: path of codebook; if it is empty, the default path codebook/FAMILY_codebook.txt will be used. For markers with multiple codebooks like AprilTag and ArUco, their default codebooks are for AprilTag (36h11) and ArUco (36h12) respectively.
  • cameraMatrix: camera intrinsic matrix, [fx, 0, cx, 0, fy, cy, 0, 0, 1].
  • distCoeffs: camera distortion coefficients (both radial and tangential), [k1, k2, p1, p2, k3, k4, k5, k6].
  • marker_size: physical size of the marker.

Besides supporting existing markers like AprilTag, ArUco, ARToolkitPlus, TopoTag & RuneTag, DeepTag also supports newly-designed markers like AprilTag-XO, AprilTag-XA and RuneTag+ (provided in folders images_tag). Set family to apriltagxo in config for AprilTag-XO and AprilTag-XA, and runetag for RuneTag+ respectively.

Terms of use

The source code is provided for research purposes only. Any commercial use is prohibited. When using the code in your research work, please cite the following paper:

"DeepTag: A General Framework for Fiducial Marker Design and Detection."
Zhuming Zhang, Yongtao Hu, Guoxing Yu, and Jingwen Dai
arXiv:2105.13731 (2021).

@article{zhang2021deeptag,
  title={{DeepTag: A General Framework for Fiducial Marker Design and Detection}},
  author={Zhang, Zhuming and Hu, Yongtao and Yu, Guoxing and Dai, Jingwen},
  year={2021},
  eprint={2105.13731},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

Contact

If you find any bug or have any question about the code, please report to the Issues page.

Comments
  • Cannot get test command to work

    Cannot get test command to work

    hi,

    would like to use your framework in a research project where i'll have a webcam tracking 20-50 tags in a scene. I tried to clone the repo, install the dependencies (although the requirements.txt file doesn't have exact package names) and when I ran the command I got:

    => python3 test_deeptag.py --config config_image.json

===========> loading model <===========
Cannot load models.
Cannot load codebook: codebook/apriltag_codebook.txt

    however, i can see that the file codebook/apriltag_codebook.txt is there along with other files in the models folder.

    Any help to get this working? Would love to try it out!

    thanks

    opened by vshesh 5
  • Unable to download data

    Unable to download data

    Dear authors of this study,

    First i would like to thank you for making your data available online, I would like to access your data,

    But since its posted in baidu drive, its not accessible outside China, can you please tell me other means to download your data.

    thanks in advance.

    opened by SujithChristopher 2
  • script for training?

    script for training?

    Hello,

    I'm doing research for drone autolanding. I would like to utilize deeptag framework, since it's framework fits me well. To briefly introduce my interest, in order to detect below image, I want to retrain deeptag network with modifying train/label dataset. image

    But I couldn't find it myself. Can you provide a code for training? If I were rude of asking it, sorry in advance.

    Thanks,

    opened by WhiteCri 2
  • Segmentation fault (core dumped) with test image

    Segmentation fault (core dumped) with test image

    hi there, I encountered segfault when running test image command.

    Below is my terminal output(I run this in docker) :

    (base) root@ea30007e7bad:~/deeptag-pytorch# python test_deeptag.py --config config_image.json ===========> loading model <===========

    Stage-1<<<<<<< 8 ROIs Stage-2<<<<<<< iter #0 iter #1 Valid ROIs: 0, 1, 2, 3, 4, 6, 7, ------timing (sec.)------ Stage-1 : [CNN 0.2850] 0.3052 Stage-2 (1 marker): [CNN 0.0291] 0.0706 Stage-2 (8 rois, 7 markers): [CNN 0.2331] 0.5320 Segmentation fault (core dumped)

    Can you give me any hink about this?

    Thanks,

    opened by WhiteCri 1
  • Reproduce training stage

    Reproduce training stage

    Hello,

    I would like to reproduce your training steps to learn your models, but I could not find a file to do it. Could you tell me how to reproduce this step ? Is this included in this project?

    Thanks for your work Best regards

    opened by DadEap 1
  • Cross shaped marker

    Cross shaped marker

    First of all, thank you for such a great work!

    I am trying to experiment detecting a cross shaped marker and I am trying to modify the codebook to detect cross markers. It would be great if you could explain the codebook text file. It has 16 columns and 587 rows, so I am not sure how it represents the marker.

    Below is the cross shaped marker I am trying to detect.

    image

    Thanks!

    opened by sangjun04 1
Owner
Yongtao Hu
Yongtao Hu
GitHub
Inference code for "StylePeople: A Generative Model of Fullbody Human Avatars" paper. This code is for the part of the paper describing video-based avatars.

NeuralTextures This is repository with inference code for paper "StylePeople: A Generative Model of Fullbody Human Avatars" (CVPR21). This code is for

Visual Understanding Lab @ Samsung AI Center Moscow 18 Oct 6, 2022
Code for paper ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loop.

Who Left the Dogs Out? Evaluation and demo code for our ECCV 2020 paper: Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization

Benjamin Biggs 29 Dec 28, 2022
The project is an official implementation of our CVPR2019 paper "Deep High-Resolution Representation Learning for Human Pose Estimation"

Deep High-Resolution Representation Learning for Human Pose Estimation (CVPR 2019) News [2020/07/05] A very nice blog from Towards Data Science introd

Leo Xiao 3.9k Jan 5, 2023
Home repository for the Regularized Greedy Forest (RGF) library. It includes original implementation from the paper and multithreaded one written in C++, along with various language-specific wrappers.

Regularized Greedy Forest Regularized Greedy Forest (RGF) is a tree ensemble machine learning method described in this paper. RGF can deliver better r

RGF-team 364 Dec 28, 2022
Official implementation of AAAI-21 paper "Label Confusion Learning to Enhance Text Classification Models"

Description: This is the official implementation of our AAAI-21 accepted paper Label Confusion Learning to Enhance Text Classification Models. The str

null 101 Nov 25, 2022
Official PyTorch implementation for paper Context Matters: Graph-based Self-supervised Representation Learning for Medical Images

Context Matters: Graph-based Self-supervised Representation Learning for Medical Images Official PyTorch implementation for paper Context Matters: Gra

null 49 Nov 23, 2022
A PyTorch re-implementation of the paper 'Exploring Simple Siamese Representation Learning'. Reproduced the 67.8% Top1 Acc on ImageNet.

Exploring simple siamese representation learning This is a PyTorch re-implementation of the SimSiam paper on ImageNet dataset. The results match that

Taojiannan Yang 72 Nov 9, 2022
Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series Forecasting.

Non-AR Spatial-Temporal Transformer Introduction Implementation of the paper NAST: Non-Autoregressive Spatial-Temporal Transformer for Time Series For

Chen Kai 66 Nov 28, 2022
This is a Pytorch implementation of the paper: Self-Supervised Graph Transformer on Large-Scale Molecular Data.

This is a Pytorch implementation of the paper: Self-Supervised Graph Transformer on Large-Scale Molecular Data.

null 212 Dec 25, 2022
Official implementation of the ICLR 2021 paper

You Only Need Adversarial Supervision for Semantic Image Synthesis Official PyTorch implementation of the ICLR 2021 paper "You Only Need Adversarial S

Bosch Research 272 Dec 28, 2022
Implementation of Nyström Self-attention, from the paper Nyströmformer

Nyström Attention Implementation of Nyström Self-attention, from the paper Nyströmformer. Yannic Kilcher video Install $ pip install nystrom-attention

Phil Wang 95 Jan 2, 2023
Implementation of SETR model, Original paper: Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.

SETR - Pytorch Since the original paper (Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers.) has no official

zhaohu xing 112 Dec 16, 2022
Official implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis https://arxiv.org/abs/2011.13775

CIPS -- Official Pytorch Implementation of the paper Image Generators with Conditionally-Independent Pixel Synthesis Requirements pip install -r requi

Multimodal Lab @ Samsung AI Center Moscow 201 Dec 21, 2022
Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

null 364 Dec 14, 2022
PyTorch implementation of paper "Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes", CVPR 2021

Neural Scene Flow Fields PyTorch implementation of paper "Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes", CVPR 20

Zhengqi Li 585 Jan 4, 2023
Implementation of Barlow Twins paper

barlowtwins PyTorch Implementation of Barlow Twins paper: Barlow Twins: Self-Supervised Learning via Redundancy Reduction This is currently a work in

IgorSusmelj 86 Dec 20, 2022
Official pytorch implementation of paper "Inception Convolution with Efficient Dilation Search" (CVPR 2021 Oral).

IC-Conv This repository is an official implementation of the paper Inception Convolution with Efficient Dilation Search. Getting Started Download Imag

Jie Liu 111 Dec 31, 2022
Official implementation of our paper "LLA: Loss-aware Label Assignment for Dense Pedestrian Detection" in Pytorch.

LLA: Loss-aware Label Assignment for Dense Pedestrian Detection This project provides an implementation for "LLA: Loss-aware Label Assignment for Dens

null 35 Dec 6, 2022
Functional TensorFlow Implementation of Singular Value Decomposition for paper Fast Graph Learning

tf-fsvd TensorFlow Implementation of Functional Singular Value Decomposition for paper Fast Graph Learning with Unique Optimal Solutions Cite If you f

Sami Abu-El-Haija 14 Nov 25, 2021