[CVPR21] LightTrack: Finding Lightweight Neural Network for Object Tracking via One-Shot Architecture Search

Overview

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search

The official implementation of the paper

LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search

Hiring research interns for visual transformer projects: [email protected]

Abstract

We present LightTrack, which uses neural architecture search (NAS) to design more lightweight and efficient object trackers. Comprehensive experiments show that our LightTrack is effective. It can find trackers that achieve superior performance compared to handcrafted SOTA trackers, such as SiamRPN++ and Ocean, while using much fewer model Flops and parameters. Moreover, when deployed on resource-constrained mobile chipsets, the discovered trackers run much faster. For example, on Snapdragon 845 Adreno GPU, LightTrack runs 12× faster than Ocean, while using 13× fewer parameters and 38× fewer Flops. Such improvements might narrow the gap between academic models and industrial deployments in object tracking task.

Environment Installation

cd lighttrack
conda create -n lighttrack python=3.6
conda activate lighttrack
bash install.sh

Data Preparation

  • Tracking Benchmarks

Please put VOT2019 dataset under $LightTrack/dataset. The prepared data should look like:

$LighTrack/dataset/VOT2019.json
$LighTrack/dataset/VOT2019/agility
$LighTrack/dataset/VOT2019/ants1
...
$LighTrack/dataset/VOT2019/list.txt

Test and evaluation

Test LightTrack-Mobile on VOT2019

bash tracking/reproduce_vot2019.sh

Flops, Params, and Speed

Compute the flops and params of our LightTrack-Mobile. The flops counter we use is pytorch-OpCounter

python tracking/FLOPs_Params.py

Test the running speed of our LightTrack-Mobile

python tracking/Speed.py
Comments
  • 论文and代码

    论文and代码

    感谢你非常棒的工作《LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Search》,但是网上好像现在找不到这篇论文,可以给个论文的下载链接吗?另外,可以尽快公布代码吗?

    opened by longmalongma 5
  • Why is a tracker based on offline tracking so robust?

    Why is a tracker based on offline tracking so robust?

    The paper is similar to siamfc++, except that the network layer is changed. Why is it so robust that it is better than online updates, such as dimp. How does this offline tracking tracker distinguish distractor?

    opened by makalo 3
  • Help needed to prepare dataset.

    Help needed to prepare dataset.

    downloaded from https://www.votchallenge.net/vot2019/dataset.html

    VOT2019 dataset preview

    but did not have list.txt

    FileNotFoundError: [Errno 2] No such file or directory: 'LightTrack/lib/utils/../../dataset/VOT2019/list.txt'

    opened by purvang3 3
  • ckpt_path is not defined ERROR

    ckpt_path is not defined ERROR

    what exactly is this path? path_name=back_04502514044521042540+cls_211000022+reg_100000111_ops_32

    On running: bash tracking/reproduce_vot2019.sh

    ===> init Siamese <==== load pretrained model from snapshot/LightTrackM/LightTrackM.pth Traceback (most recent call last): File "tracking/test_lighttrack.py", line 181, in main() File "tracking/test_lighttrack.py", line 165, in main siam_net = load_pretrain(siam_net, args.resume) File "/home/nishat/adasi/tracking/LightTrack/tracking/../lib/utils/utils.py", line 702, in load_pretrain ckpt = torch.load(ckpt_path, map_location=lambda storage, loc: storage(device))
    NameError: name 'ckpt_path' is not defined VOT2019 loading VOT2019: 100%|██████████████████████████████████| 60/60 [00:00<00:00, 61.31it/s, zebrafish1] eval ar: 100%|████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 15.58it/s] eval eao: 0%| | 0/1 [00:00<?, ?it/s] multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/nishat/miniconda3/envs/lighttrack/lib/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/home/nishat/adasi/tracking/LightTrack/lib/eval_toolkit/bin/../../eval_toolkit/pysot/evaluation/eao_benchmark.py", line 47, in eval eao = self._calculate_eao(tracker_name, self.tags) File "/home/nishat/adasi/tracking/LightTrack/lib/eval_toolkit/bin/../../eval_toolkit/pysot/evaluation/eao_benchmark.py", line 110, in _calculate_eao max_len = max([len(x) for x in all_overlaps]) ValueError: max() arg is an empty sequence """

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last): File "lib/eval_toolkit/bin/eval.py", line 148, in trackers), desc='eval eao', total=len(trackers), ncols=100): File "/home/nishat/miniconda3/envs/lighttrack/lib/python3.6/site-packages/tqdm/std.py", line 1178, in iter for obj in iterable: File "/home/nishat/miniconda3/envs/lighttrack/lib/python3.6/multiprocessing/pool.py", line 735, in next raise value ValueError: max() arg is an empty sequence

    opened by nishathussain 2
  • About inference speed

    About inference speed

    Hello ,your work is very wonderful and testing in my video achieved a good performance. One frame inference time is about 13ms and ocean is about 15ms in 2080Ti. How to raise the fps? Thanks for your open-source codes and methods and look forward your prompt reply.

    opened by Youjiangbaba 2
  • About the pixel_corr_mat method

    About the pixel_corr_mat method

    I noticed that the method of cross-correlation is different from the general siam tracker. In the code, the matrix multiplication method is used as the pixel cross-correlation, and the channel separable convolution method is generally used as the channel cross-correlation. What are the advantages of using pixel correlation? Looking forward to your reply, thank you very much!

    opened by TCBocean 2
  • Test and Evaluation issue

    Test and Evaluation issue

    After testing your work on UAV123 dataset and evaluating the result with OPE benchmark the result is have very low accuracy Is there any wrong in your code or paper .....

    opened by abbasaqeel 0
  • Another issue about backbone supernet pre-training

    Another issue about backbone supernet pre-training

    Hi, i read the paper again, in section 4.3, it confuse me that how can you make sure that the best backbone architecture(path) can be sampled in the pre-training stage and the sampled path is different from the previous paths? Since there are about 78 billion possible backbones(paths), and i think it's impossible to pre-train them all and maybe the actual best backbone(path) was not been sampled, so what if this situation was really happend during your expirement? Thanks, looking for your reply.

    opened by Someone-LikeU 0
  • Issue about backbone supernet pre-training

    Issue about backbone supernet pre-training

    Hi, it's a very interesting and excellent work, and I am a beginner in NAS, I'm very interesting in the pre-training of backbone supernet, but I didn't find this part in your released code, did you not release this part or it's just my fault? Thanks, looking for your reply.

    opened by Someone-LikeU 0
Owner
Multimedia Research
Multimedia Research at Microsoft Research Asia
Multimedia Research
PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation

PocketNet This is the official repository of the paper: PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and M

Fadi Boutros 40 Dec 22, 2022
Released code for Objects are Different: Flexible Monocular 3D Object Detection, CVPR21

MonoFlex Released code for Objects are Different: Flexible Monocular 3D Object Detection, CVPR21. Work in progress. Installation This repo is tested w

Yunpeng 169 Dec 6, 2022
code for paper "Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?"

Does Unsupervised Architecture Representation Learning Help Neural Architecture Search? Code for paper: Does Unsupervised Architecture Representation

null 39 Dec 17, 2022
FairMOT - A simple baseline for one-shot multi-object tracking

FairMOT - A simple baseline for one-shot multi-object tracking

Yifu Zhang 3.6k Jan 8, 2023
A "gym" style toolkit for building lightweight Neural Architecture Search systems

A "gym" style toolkit for building lightweight Neural Architecture Search systems

Jack Turner 12 Nov 5, 2022
Few-shot Neural Architecture Search

One-shot Neural Architecture Search uses a single supernet to approximate the performance each architecture. However, this performance estimation is super inaccurate because of co-adaption among operations in supernet.

Yiyang Zhao 38 Oct 18, 2022
Densely Connected Search Space for More Flexible Neural Architecture Search (CVPR2020)

DenseNAS The code of the CVPR2020 paper Densely Connected Search Space for More Flexible Neural Architecture Search. Neural architecture search (NAS)

Jamin Fong 291 Nov 18, 2022
code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

code for "AttentiveNAS Improving Neural Architecture Search via Attentive Sampling"

Facebook Research 94 Oct 26, 2022
PyTorch implementation of "Efficient Neural Architecture Search via Parameters Sharing"

Efficient Neural Architecture Search (ENAS) in PyTorch PyTorch implementation of Efficient Neural Architecture Search via Parameters Sharing. ENAS red

Taehoon Kim 2.6k Dec 31, 2022
The official repo for CVPR2021——ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search.

ViPNAS: Efficient Video Pose Estimation via Neural Architecture Search [paper] Introduction This is the official implementation of ViPNAS: Efficient V

Lumin 42 Sep 26, 2022
[CVPR 2021] 'Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator'

[CVPR2021] Searching by Generating: Flexible and Efficient One-Shot NAS with Architecture Generator Overview This is the entire codebase for the paper

null 35 Dec 1, 2022
Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

HLA-Face: Joint High-Low Adaptation for Low Light Face Detection The official PyTorch implementation for HLA-Face: Joint High-Low Adaptation for Low L

Wenjing Wang 77 Dec 8, 2022
Repository relating to the CVPR21 paper TimeLens: Event-based Video Frame Interpolation

TimeLens: Event-based Video Frame Interpolation This repository is about the High Speed Event and RGB (HS-ERGB) dataset, used in the 2021 CVPR paper T

Robotics and Perception Group 544 Dec 19, 2022
Official repository for CVPR21 paper "Deep Stable Learning for Out-Of-Distribution Generalization".

StableNet StableNet is a deep stable learning method for out-of-distribution generalization. This is the official repo for CVPR21 paper "Deep Stable L

null 120 Dec 28, 2022
DeepHyper: Scalable Asynchronous Neural Architecture and Hyperparameter Search for Deep Neural Networks

What is DeepHyper? DeepHyper is a software package that uses learning, optimization, and parallel computing to automate the design and development of

DeepHyper Team 214 Jan 8, 2023
Model search is a framework that implements AutoML algorithms for model architecture search at scale

Model search (MS) is a framework that implements AutoML algorithms for model architecture search at scale. It aims to help researchers speed up their exploration process for finding the right model architecture for their classification problems (i.e., DNNs with different types of layers).

Google 3.2k Dec 31, 2022
Official PyTorch implementation of Joint Object Detection and Multi-Object Tracking with Graph Neural Networks

This is the official PyTorch implementation of our paper: "Joint Object Detection and Multi-Object Tracking with Graph Neural Networks". Our project website and video demos are here.

Richard Wang 443 Dec 6, 2022
A object detecting neural network powered by the yolo architecture and leveraging the PyTorch framework and associated libraries.

Yolo-Powered-Detector A object detecting neural network powered by the yolo architecture and leveraging the PyTorch framework and associated libraries

Luke Wilson 1 Dec 3, 2021