STMTrack
This is the official implementation of the paper: STMTrack: Template-free Visual Tracking with Space-time Memory Networks.
Setup
-
Prepare Anaconda, CUDA and the corresponding toolkits. CUDA version required: 10.0+
-
Create a new conda environment and activate it.
conda create -n STMTrack python=3.7 -y
conda activate STMTrack
- Install
pytorch
andtorchvision
.
conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.0 -c pytorch
# pytorch v1.5.0, v1.6.0, or higher should also be OK.
- Install other required packages.
pip install -r requirements.txt
Test
- Prepare the datasets: OTB2015, VOT2018, UAV123, GOT-10k, TrackingNet, LaSOT, ILSVRC VID*, ILSVRC DET*, COCO*, and something else you want to test. Set the paths as the following:
├── STMTrack
| ├── ...
| ├── ...
| ├── datasets
| | ├── COCO -> /opt/data/COCO
| | ├── GOT-10k -> /opt/data/GOT-10k
| | ├── ILSVRC2015 -> /opt/data/ILSVRC2015
| | ├── LaSOT -> /opt/data/LaSOT/LaSOTBenchmark
| | ├── OTB
| | | └── OTB2015 -> /opt/data/OTB2015
| | ├── TrackingNet -> /opt/data/TrackingNet
| | ├── UAV123 -> /opt/data/UAV123/UAV123
| | ├── VOT
| | | ├── vot2018
| | | | ├── VOT2018 -> /opt/data/VOT2018
| | | | └── VOT2018.json
- Notes
i. Star notation(*): just for training. You can ignore these datasets if you just want to test the tracker.
ii. In this case, we create soft links for every dataset. The real storage location of all datasets is
/opt/data/
. You can change them according to your situation.iii. The
VOT2018.json
file can be download from here.
-
Download the models we trained.
-
Use the path of the trained model to set the
pretrain_model_path
item in the configuration file correctly, then run the shell command. -
Note that all paths we used here are relative, not absolute. See any configuration file in the
experiments
directory for examples and details.
General command format
python main/test.py --config testing_dataset_config_file_path
Take GOT-10k as an example:
python main/test.py --config experiments/stmtrack/test/got10k/stmtrack-googlenet-got.yaml
Training
- Prepare the datasets as described in the last subsection.
- Download the pretrained backbone model from here.
- Run the shell command.
training based on the GOT-10k benchmark
python main/train.py --config experiments/stmtrack/train/got10k/stmtrack-googlenet-trn.yaml
training with full data
python main/train.py --config experiments/stmtrack/train/fulldata/stmtrack-googlenet-trn-fulldata.yaml
Testing Results
Click here to download all the following.
- OTB2015
- GOT-10k
- LaSOT
- TrackingNet
- UAV123
- TNL2K
- evaluated by @Xiao Wang.
- The results can be downloaded from Google Drive. See issue #2 for more details.
Acknowledgement
Repository
This repository is developed based on the single object tracking framework video_analyst. See it for more instructions and details.
References
@inproceedings{fu2021stmtrack,
title={STMTrack: Template-free Visual Tracking with Space-time Memory Networks},
author={Fu, Zhihong and Liu, Qingjie and Fu, Zehua and Wang, Yunhong},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={13774--13783},
year={2021}
}
Contact
- Zhihong Fu@fzh0917
If you have any questions, just create issues or email me