PySOT - SenseTime Research platform for single object tracking, implementing algorithms like SiamRPN and SiamMask.

Related tags

Deep Learning tracking computer-vision siamese-network
Overview

PySOT

PySOT is a software system designed by SenseTime Video Intelligence Research team. It implements state-of-the-art single object tracking algorithms, including SiamRPN and SiamMask. It is written in Python and powered by the PyTorch deep learning framework. This project also contains a Python port of toolkit for evaluating trackers.

PySOT has enabled research projects, including: SiamRPN, DaSiamRPN, SiamRPN++, and SiamMask.

Example SiamFC, SiamRPN and SiamMask outputs.

Introduction

The goal of PySOT is to provide a high-quality, high-performance codebase for visual tracking research. It is designed to be flexible in order to support rapid implementation and evaluation of novel research. PySOT includes implementations of the following visual tracking algorithms:

using the following backbone network architectures:

Additional backbone architectures may be easily implemented. For more details about these models, please see References below.

Evaluation toolkit can support the following datasets:

📎 OTB2015 📎 VOT16/18/19 📎 VOT18-LT 📎 LaSOT 📎 UAV123

Model Zoo and Baselines

We provide a large set of baseline results and trained models available for download in the PySOT Model Zoo.

Installation

Please find installation instructions for PyTorch and PySOT in INSTALL.md.

Quick Start: Using PySOT

Add PySOT to your PYTHONPATH

export PYTHONPATH=/path/to/pysot:$PYTHONPATH

Download models

Download models in PySOT Model Zoo and put the model.pth in the correct directory in experiments

Webcam demo

python tools/demo.py \
    --config experiments/siamrpn_r50_l234_dwxcorr/config.yaml \
    --snapshot experiments/siamrpn_r50_l234_dwxcorr/model.pth
    # --video demo/bag.avi # (in case you don't have webcam)

Download testing datasets

Download datasets and put them into testing_dataset directory. Jsons of commonly used datasets can be downloaded from Google Drive or BaiduYun. If you want to test tracker on new dataset, please refer to pysot-toolkit to setting testing_dataset.

Test tracker

cd experiments/siamrpn_r50_l234_dwxcorr
python -u ../../tools/test.py 	\
	--snapshot model.pth 	\ # model path
	--dataset VOT2018 	\ # dataset name
	--config config.yaml	  # config file

The testing results will in the current directory(results/dataset/model_name/)

Eval tracker

assume still in experiments/siamrpn_r50_l234_dwxcorr_8gpu

python ../../tools/eval.py 	 \
	--tracker_path ./results \ # result path
	--dataset VOT2018        \ # dataset name
	--num 1 		 \ # number thread to eval
	--tracker_prefix 'model'   # tracker_name

Training 🔧

See TRAIN.md for detailed instruction.

Getting Help 🔨

If you meet problem, try searching our GitHub issues first. We intend the issues page to be a forum in which the community collectively troubleshoots problems. But please do not post duplicate issues. If you have similar issue that has been closed, you can reopen it.

  • ModuleNotFoundError: No module named 'pysot'

🎯 Solution: Run export PYTHONPATH=path/to/pysot first before you run the code.

  • ImportError: cannot import name region

🎯 Solution: Build region by python setup.py build_ext —-inplace as decribled in INSTALL.md.

References

Contributors

License

PySOT is released under the Apache 2.0 license.

Comments
  • training

    training

    请问如何不通过命令行运行train.py?目前我直接在运行test.py 可以，但是直接运行train.py: File "/home/public/anaconda3/lib/python3.6/os.py", line 669, in getitem raise KeyError(key) from None KeyError: 'RANK'

    请问如何在代码中设置： CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 python -m torch.distributed.launch
    --nproc_per_node=8
    --master_port=2333
    ../../tools/train.py --cfg config.yaml 中的值，使得可以直接在pycharm运行train.py， 谢谢您的回复。

    opened by ghost 53
  • pysot model.pth to onnx occurred error

    pysot model.pth to onnx occurred error

    torch_out = torch.onnx.export(net,dummy_input,"siamrpn_alex_dwxcorr.pth",export_params=True) when export siamrpn_alex_dwxcorr.pth to onnx the error is occurred.

    error position-> :template = data['template'].cuda()

    builtins.IndexError: too many indices for tensor of dimension 4

    why this happened and how to solve it

    opened by rollben 30
  • 我制作了一个只有30张的coco数据集，用于训练但还是以20/485000的进度要跑很久 I made a coco dataset with only 30 sheets for training but still run for a long time at 20/485000

    我制作了一个只有30张的coco数据集，用于训练但还是以20/485000的进度要跑很久 I made a coco dataset with only 30 sheets for training but still run for a long time at 20/485000

    一开始使用的是readme里下载的coco数据集，cfg里相关参数也进行了修改，但进度process一直是20/485000慢慢移，后面想看跑完整个训练的效果就换了自己的数据集改成的coco格式，1900张左右也还是一样，换成30张也是一样，看代码应该是dataset.py的原因，请问如何修改能使process进度正常？

    opened by jiayuan-running 23
  • Problem? when trained on vid, det, coco and youtubebb, got my EAO=0.164 in epoch 19

    Problem? when trained on vid, det, coco and youtubebb, got my EAO=0.164 in epoch 19

    Can you tell me what problem? when trained siamrpn_r50_l234_dwxcorr_gpu on vid, det, coco and youtubebb, i only got EAO=0.164 in epoch 19 in VOT2018. EAO=0.202 in epoch 14 in VOT2018. i use siamrpn_r50_l234_dwxcorr_gpu to train, only modify row 136 in config.py _C.DATASET.NAMES = ('VID', 'COCO', 'DET', 'YOUTUBEBB', 'MOT') to _C.DATASET.NAMES = ('VID', 'COCO', 'DET', 'YOUTUBEBB')

    opened by Senhui-Qiu 19
  • AttributeError: 'NoneType' object has no attribute 'shape'

    AttributeError: 'NoneType' object has no attribute 'shape'

    I am only using coco 2014 datasets for training. I have done crop and generate data info ( train2014.json , val2014json, val.json, train.json ) but I got this error if it image path problem where to fix it, I checked dataset.py as well.

    File "./tools/train.py", line 319, in main() File "./tools/train.py", line 314, in main train(train_loader, dist_model, optimizer, lr_scheduler, tb_writer) File "./tools/train.py", line 176, in train for idx, data in enumerate(train_loader): File "/home/thomas/anaconda3/envs/pysot/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 336, in next return self._process_next_batch(batch) File "/home/thomas/anaconda3/envs/pysot/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 357, in _process_next_batch raise batch.exc_type(batch.exc_msg) AttributeError: Traceback (most recent call last): File "/home/thomas/anaconda3/envs/pysot/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 106, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/thomas/anaconda3/envs/pysot/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 106, in samples = collate_fn([dataset[i] for i in batch_indices]) File "/home/thomas/pysot/pysot/datasets/dataset.py", line 253, in getitem template_box = self._get_bbox(template_image, template[1]) File "/home/thomas/pysot/pysot/datasets/dataset.py", line 214, in _get_bbox imh, imw = image.shape[:2] AttributeError: 'NoneType' object has no attribute 'shape'

    These datasets are inside /pysot/training_dataset/coco: (which has crop511, train2014 and val2014 folder of images)

    opened by ThomasBomjan 17
  • Training issues

    Training issues

    Thank you for your work, Could I run train.py in single GPU?what should I do? When I run 'train.py' directly, I get the following error:Traceback (most recent call last): File "../../tools/train.py", line 314, in main() File "../../tools/train.py", line 256, in main rank, world_size = dist_init() File "/home/db/Subject/pysot/pysot/utils/distributed.py", line 104, in dist_init rank, world_size = _dist_init() File "/home/db/Subject/pysot/pysot/utils/distributed.py", line 83, in _dist_init rank = int(os.environ['RANK']) File "/home/db/anaconda3/envs/pysot/lib/python3.7/os.py", line 678, in getitem raise KeyError(key) from None KeyError: 'RANK'

    opened by ZZXin 16
  • KeyError: 'epoch'

    KeyError: 'epoch'

    [2019-07-08 13:15:21,166-rk0-train.py#370] resume from /home/cbl/PycharmProjects/pysot-master/experiments/siamrpn_r50_l234_dwxcorr/model.pth Traceback (most recent call last): File "../../tools/train.py", line 386, in main() File "../../tools/train.py", line 374, in main restore_from(model, optimizer, cfg.TRAIN.RESUME) File "/home/cbl/PycharmProjects/pysot-master-train/pysot/utils/model_load.py", line 77, in restore_from Traceback (most recent call last): File "../../tools/train.py", line 386, in epoch = ckpt['epoch'] KeyError: 'epoch'

    opened by ghost 15
  • MobilenetV2 config file. Can you share the mobilenetv2 training config.

    MobilenetV2 config file. Can you share the mobilenetv2 training config.

    Which layer should I train in backbone net？ In pysot/core/config，only resnet50 backbone train layer provided. For Mobilenetv2, which layer should I train? Thank you.

    enhancement 
    opened by SteveTsg 14
  • RuntimeError: Error(s) in loading state_dict for ModelBuilder

    RuntimeError: Error(s) in loading state_dict for ModelBuilder

    May I ask how to solve this error when running the demo.py. File "/home/shiyuanyuan/anaconda3/envs/pysot/lib/python3.7/site-packages/torch/nn/modules/module.py", line 719, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for ModelBuilder: Missing key(s) in state_dict:

    opened by fxlong 12
  • > 我们假设追踪的是一个刚体，即其尺寸、形状在短时间内不会发生较大的变动。因此，我们对这样的改变进行惩罚。

    > 我们假设追踪的是一个刚体，即其尺寸、形状在短时间内不会发生较大的变动。因此，我们对这样的改变进行惩罚。

    我们假设追踪的是一个刚体，即其尺寸、形状在短时间内不会发生较大的变动。因此，我们对这样的改变进行惩罚。

    您好，我想问下，在tracking阶段，得到的bbox的位置不就可以了吗？为什么还要加上模板的中心点坐标位置呢？

    cx = bbox[0] + self.center_pos[0] #-15+345 ,模板的中心点
cy = bbox[1] + self.center_pos[1]

    Originally posted by @buaacarzp in https://github.com/STVIR/pysot/issues/330#issuecomment-601972967

    opened by buaacarzp 11
  • conda create --name pysot python=3.7 conda activate pysot

    conda create --name pysot python=3.7 conda activate pysot

    why I run it and false conda create --name pysot python=3.7 conda activate pysot

    CondaHTTPError: HTTP 404 NOT FOUND for url https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/noarch/repodata.json Elapsed: 00:09.424438

    The remote server could not find the noarch directory for the requested channel with url: https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge

    As of conda 4.3, a valid channel must contain a noarch/repodata.json and associated noarch/repodata.json.bz2 file, even if noarch/repodata.json is empty. please request that the channel administrator create noarch/repodata.json and associated noarch/repodata.json.bz2 files. $ mkdir noarch $ echo '{}' > noarch/repodata.json $ bzip2 -k noarch/repodata.json

    You will need to adjust your conda configuration to proceed. Use conda config --show channels to view your configuration's current state. Further configuration help can be found at https://conda.io/docs/config.html.

    opened by ghost 11
  • Validation loss doesn't decrease (驗證損失不會下降)

    Validation loss doesn't decrease (驗證損失不會下降)

    When training, I found although the training loss is decreasing, the validation loss doesn't decrease. But the latter model is indeed better than the former. Does anyone face the same weird phenomenon?

    我在訓練時發現，雖然訓練損失有在下降，但是驗證損失卻幾乎沒下降， 但奇怪的是，訓練越多次的 model 的確比前面的 model 表現更好？ 有人也遇到類似的奇怪現象嗎？

    要怎麼解釋這種情況呢？

    opened by AdrienGuo 0
  • Pretrained resnet50 backbone with classification layer

    Pretrained resnet50 backbone with classification layer

    Could you provide the pretrained resnet50 model (https://drive.google.com/drive/folders/1DuXVWVYIeynAcvt9uxtkuleV6bs6e3T9) with the classification layer that you needed for training on ImageNet?

    opened by al-ludwig 0
  • 分布式训练卡住了 如下图所示

    分布式训练卡住了 如下图所示

    [2022-08-31 16:56:10,107-rk0-model_load.py# 42] remove prefix 'module.' [2022-08-31 16:56:10,108-rk0-model_load.py# 33] used keys:265 [2022-08-31 16:56:10,116-rk0-train.py# 58] build train dataset [2022-08-31 16:56:10,117-rk0-dataset.py# 39] loading VID [2022-08-31 16:56:10,117-rk0-dataset.py# 64] VID loaded

    命令为： CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch
    --nproc_per_node=2
    --master_port=2333
    ../../tools/train.py --cfg config.yaml

    opened by vousemevoyez1024 0
  • demo.py运行问题

    demo.py运行问题

    系统windows 10，按照教程安装运行demo.py，程序无报错，但无法显示跟踪操作平台，亦没有： Select a ROI and then press SPACE or ENTER button! Cancel the selection process by pressing c button!显示。 像是卡死了一样。请问是什么问题？

    opened by jklkid 1
Owner
STVIR
SenseTime Video Intelligence Research Team
STVIR
GitHub
An integration of several popular automatic augmentation methods, including OHL (Online Hyper-Parameter Learning for Auto-Augmentation Strategy) and AWS (Improving Auto Augment via Augmentation Wise Weight Sharing) by Sensetime Research.

An integration of several popular automatic augmentation methods, including OHL (Online Hyper-Parameter Learning for Auto-Augmentation Strategy) and AWS (Improving Auto Augment via Augmentation Wise Weight Sharing) by Sensetime Research.

null 45 Dec 8, 2022
Python package for multiple object tracking research with focus on laboratory animals tracking.

motutils is a Python package for multiple object tracking research with focus on laboratory animals tracking. Features loads: MOTChallenge CSV, sleap

Matěj Šmíd 2 Sep 5, 2022
1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

1st place solution to the Satellite Image Change Detection Challenge hosted by SenseTime

Lihe Yang 209 Jan 1, 2023
FastReID is a research platform that implements state-of-the-art re-identification algorithms.

FastReID is a research platform that implements state-of-the-art re-identification algorithms.

JDAI-CV 2.8k Jan 7, 2023
Implementing yolov4 target detection and tracking based on nao robot

Implementing yolov4 target detection and tracking based on nao robot

null 6 Apr 19, 2022
This is an open source library implementing hyperbox-based machine learning algorithms

hyperbox-brain is a Python open source toolbox implementing hyperbox-based machine learning algorithms built on top of scikit-learn and is distributed

Complex Adaptive Systems (CAS) Lab - University of Technology Sydney 21 Dec 14, 2022
BisQue is a web-based platform designed to provide researchers with organizational and quantitative analysis tools for 5D image data. Users can extend BisQue by implementing containerized ML workflows.

Overview BisQue is a web-based platform specifically designed to provide researchers with organizational and quantitative analysis tools for up to 5D

Vision Research Lab @ UCSB 26 Nov 29, 2022
The official implementation of ICCV paper "Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds".

Box-Aware Tracker (BAT) Pytorch-Lightning implementation of the Box-Aware Tracker. Box-Aware Feature Enhancement for Single Object Tracking on Point C

Kangel Zenn 5 Mar 26, 2022
A simple implementation of Kalman filter in single object tracking

kalman-filter-in-single-object-tracking A simple implementation of Kalman filter in single object tracking https://www.bilibili.com/video/BV1Qf4y1J7D4

null 130 Dec 26, 2022
Python Single Object Tracking Evaluation

pysot-toolkit The purpose of this repo is to provide evaluation API of Current Single Object Tracking Dataset, including VOT2016 VOT2018 VOT2018-LT OT

null 348 Dec 22, 2022
A general python framework for single object tracking in LiDAR point clouds, based on PyTorch Lightning.

Open3DSOT A general python framework for single object tracking in LiDAR point clouds, based on PyTorch Lightning. The official code release of BAT an

Kangel Zenn 172 Dec 23, 2022
Object tracking and object detection is applied to track golf puts in real time and display stats/games.

Putting_Game Object tracking and object detection is applied to track golf puts in real time and display stats/games. Works best with the Perfect Prac

Max 1 Dec 29, 2021
Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

This is the official PyTorch implementation of our paper: "Joint Object Detection and Multi-Object Tracking with Graph Neural Networks". Our project website and video demos are here.

Richard Wang 443 Dec 6, 2022
Object Detection and Multi-Object Tracking

Object Detection and Multi-Object Tracking

Bobby Chen 1.6k Jan 4, 2023
SiamMOT is a region-based Siamese Multi-Object Tracking network that detects and associates object instances simultaneously.

SiamMOT: Siamese Multi-Object Tracking

null 432 Dec 17, 2022
TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction TSDF++ is a novel multi-object TSDF formulation that can encode mult

ETHZ ASL 130 Dec 29, 2022
Joint detection and tracking model named DEFT, or ``Detection Embeddings for Tracking.

DEFT: Detection Embeddings for Tracking DEFT: Detection Embeddings for Tracking, Mohamed Chaabane, Peter Zhang, J. Ross Beveridge, Stephen O'Hara

Mohamed Chaabane 253 Dec 18, 2022
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-t

Facebook Research 5.1k Jan 4, 2023
Hand gesture recognition based whiteboard that allows you to write on live webcam. This is the first version and has features like 4 different colors, eraser and a recording option that records your session and saves it in a "recordings" folder. Use index finger to draw and two or more fingers to move around and select items. Future version will contain more functionalities like changeable thickness, color palette, integration with zoom and google meet etc.

hand-write Hand gesture recognition based whiteboard that allows you to write on live webcam. This is the first version and has features like 4 differ

null 27 Dec 16, 2022