Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022)

Related tags

Deep Learning UDAT
Overview

Unsupervised Domain Adaptation for Nighttime Aerial Tracking (CVPR2022)

Junjie Ye, Changhong Fu, Guangze Zheng, Danda Pani Paudel, and Guang Chen. Unsupervised Domain Adaptation for Nighttime Aerial Tracking. In CVPR, pages 1-10, 2022.

featured

Overview

UDAT is an unsupervised domain adaptation framework for visual object tracking. This repo contains its Python implementation.

Paper | NAT2021 benchmark

Testing UDAT

1. Preprocessing

Before training, we need to preprocess the unlabelled training data to generate training pairs.

  1. Download the proposed NAT2021-train set

  2. Customize the directory of the train set in lowlight_enhancement.py and enhance the nighttime sequences

    cd preprocessing/
    python lowlight_enhancement.py # enhanced sequences will be saved at '/YOUR/PATH/NAT2021/train/data_seq_enhanced/'
  3. Download the video saliency detection model here and place it at preprocessing/models/checkpoints/.

  4. Predict salient objects and obtain candidate boxes

    python inference.py # candidate boxes will be saved at 'coarse_boxes/' as .npy
  5. Generate pseudo annotations from candidate boxes using dynamic programming

    python gen_seq_bboxes.py # pseudo box sequences will be saved at 'pseudo_anno/'
  6. Generate cropped training patches and a JSON file for training

    python par_crop.py
    python gen_json.py

2. Train

Take UDAT-CAR for instance.

  1. Apart from above target domain dataset NAT2021, you need to download and prepare source domain datasets VID and GOT-10K.

  2. Download the pre-trained daytime model (SiamCAR/SiamBAN) and place it at UDAT/tools/snapshot.

  3. Start training

    cd UDAT/CAR
    export PYTHONPATH=$PWD
    python tools/train.py

3. Test

Take UDAT-CAR for instance.

  1. For quick test, you can download our trained model for UDAT-CAR (or UDAT-BAN) and place it at UDAT/CAR/experiments/udatcar_r50_l234.

  2. Start testing

    python tools/test.py --dataset NAT

4. Eval

  1. Start evaluating
    python tools/eval.py --dataset NAT

Demo

Demo video

Reference

@Inproceedings{Ye2022CVPR,

title={{Unsupervised Domain Adaptation for Nighttime Aerial Tracking}},

author={Ye, Junjie and Fu, Changhong and Zheng, Guangze and Paudel, Danda Pani and Chen, Guang},

booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},

year={2022},

pages={1-10}

}

Acknowledgments

We sincerely thank the contribution of following repos: SiamCAR, SiamBAN, DCFNet, DCE, and USOT.

Contact

If you have any questions, please contact Junjie Ye at [email protected] or Changhong Fu at [email protected].

You might also like...
IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)
IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)

This repo is the official implementation of our paper "Instance Adaptive Self-training for Unsupervised Domain Adaptation". The purpose of this repo is to better communicate with you and respond to your questions. This repo is almost the same with Another-Version, and you can also refer to that version.

Code of TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation

TVT Code of TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation Datasets: Digit: MNIST, SVHN, USPS Object: Office, Office-Home, Vi

Code for CVPR2021 "Visualizing Adapted Knowledge in Domain Transfer". Visualization for domain adaptation. #explainable-ai

Visualizing Adapted Knowledge in Domain Transfer @inproceedings{hou2021visualizing, title={Visualizing Adapted Knowledge in Domain Transfer}, auth

[CVPR2021] Domain Consensus Clustering for Universal Domain Adaptation

[CVPR2021] Domain Consensus Clustering for Universal Domain Adaptation [Paper] Prerequisites To install requirements: pip install -r requirements.txt

A Pytorch Implementation of [Source data‐free domain adaptation of object detector through domain

A Pytorch Implementation of Source data‐free domain adaptation of object detector through domain‐specific perturbation Please follow Faster R-CNN and

HiFT: Hierarchical Feature Transformer for Aerial Tracking (ICCV2021)
HiFT: Hierarchical Feature Transformer for Aerial Tracking (ICCV2021)

HiFT: Hierarchical Feature Transformer for Aerial Tracking Ziang Cao, Changhong Fu, Junjie Ye, Bowen Li, and Yiming Li Our paper is Accepted by ICCV 2

Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark
Official code for 'Robust Siamese Object Tracking for Unmanned Aerial Manipulator' and offical introduction to UAMT100 benchmark

SiamSA: Robust Siamese Object Tracking for Unmanned Aerial Manipulator Demo video 📹 Our video on Youtube and bilibili demonstrates the evaluation of

IJCAI2020 & IJCV 2020 :city_sunrise: Unsupervised Scene Adaptation with Memory Regularization in vivo
IJCAI2020 & IJCV 2020 :city_sunrise: Unsupervised Scene Adaptation with Memory Regularization in vivo

Seg_Uncertainty In this repo, we provide the code for the two papers, i.e., MRNet:Unsupervised Scene Adaptation with Memory Regularization in vivo, IJ

Progressive Domain Adaptation for Object Detection
Progressive Domain Adaptation for Object Detection

Progressive Domain Adaptation for Object Detection Implementation of our paper Progressive Domain Adaptation for Object Detection, based on pytorch-fa

Comments
  • Why the 'DDP' mode cannot work on this framework

    Why the 'DDP' mode cannot work on this framework

    Hi, I try to train UDAT with 4 2080 Ti. However, dp mode causes uneven distribution of GPU memory. For example, only the main GPU occupies 11G, while the remaining three occupy only 6G on average. I change it to DDP mode but it doesn't work. Would you have any ideas?

    Besides, I find several bugs.

    1. in the train.py, when calculating the summation of discriminator output, the numpy function cannot operate on tensors with grad. i.e., train.py line 257-258, 285-286, 297-298
    D_out_z = np.sum([Disc(F.softmax(_zf_up_t, dim=1)) for _zf_up_t in zf_up_t])/3.0
    D_out_x = np.sum([Disc(F.softmax(_xf_up_t, dim=1)) for _xf_up_t in xf_up_t])/3.0
    

    I think it should be

    D_out_z = torch.stack([Disc(F.softmax(_zf_up_t, dim=1)) for _zf_up_t in zf_up_t]).sum(0) / 3.
    D_out_x = torch.stack([Disc(F.softmax(_xf_up_t, dim=1)) for _xf_up_t in xf_up_t]).sum(0) / 3.
    
    1. in the eval.py, the line 67 elif 'NAT' in args.dataset: should be elif 'NAT' == args.dataset:. Otherwise, the results of NAT_L would also go into this branch.
    opened by kongbia 10
  • The code of t-SNE in Figure 4

    The code of t-SNE in Figure 4

    Dear author, thank you very much for your work. I wonder if the t-SNE code used in Figure 4 will be made public? This visualization method is very interesting! Thanks in advance!

    t-SNE for Unsupervised Domain Adaptation for Nighttime Aerial Tracking

    opened by xyl-507 2
  • About val dataset and the result of two baselines

    About val dataset and the result of two baselines

    Hi. I have two questions:

    1. It seems that there is no val dataset in NAT, so do you directly use the model saved in the last epoch to test?
    2. Since the train set of NAT does not have labels and the two baselines(SiamCAR & SiamBAN) are supervised methods, I wonder whether the results of two baselines used to compare with UDAT are only trained on GOT-10K and VID.
    opened by cjyiiiing 2
  • question about  transformer bridging layer

    question about transformer bridging layer

    Hello, I'm very interested in your research. Because the datasets are too large, it is not easy to run through the code. I want to know the size of the feature extractor when it is passed into the transformer bridging layer. Does the feature size change after feature alignment?

    opened by WANGLEKV 1
Owner
Intelligent Vision for Robotics in Complex Environment
Adaptive Vision for Robotics in Complex Environment
Intelligent Vision for Robotics in Complex Environment
TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022)

TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022) Ziang Cao and Ziyuan Huang and Liang Pan and Shiwei Zhang and Ziwei Liu and Changhong Fu In

Intelligent Vision for Robotics in Complex Environment 100 Dec 19, 2022
Adversarial Adaptation with Distillation for BERT Unsupervised Domain Adaptation

Knowledge Distillation for BERT Unsupervised Domain Adaptation Official PyTorch implementation | Paper Abstract A pre-trained language model, BERT, ha

Minho Ryu 29 Nov 30, 2022
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation

CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation [arxiv] This is the official repository for CDTrans: Cross-domain Transformer for

null 238 Dec 22, 2022
CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation

[ICCV2021] TransReID: Transformer-based Object Re-Identification [pdf] The official repository for TransReID: Transformer-based Object Re-Identificati

DamoCV 569 Dec 30, 2022
Aerial Imagery dataset for fire detection: classification and segmentation (Unmanned Aerial Vehicle (UAV))

Aerial Imagery dataset for fire detection: classification and segmentation using Unmanned Aerial Vehicle (UAV) Title FLAME (Fire Luminosity Airborne-b

null 79 Jan 6, 2023
code for our paper "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer"

SHOT++ Code for our TPAMI submission "Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer" that is ext

null 75 Dec 16, 2022
The official codes of "Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners".

SSL models are Strong UDA learners Introduction This is the official code of paper "Semi-supervised Models are Strong Unsupervised Domain Adaptation L

Yabin Zhang 26 Dec 26, 2022
A PyTorch implementation for Unsupervised Domain Adaptation by Backpropagation(DANN), support Office-31 and Office-Home dataset

DANN A PyTorch implementation for Unsupervised Domain Adaptation by Backpropagation Prerequisites Linux or OSX NVIDIA GPU + CUDA (may CuDNN) and corre

null 8 Apr 16, 2022
Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, Pattern Recognition

USDAN The implementation of Unified unsupervised and semi-supervised domain adaptation network for cross-scenario face anti-spoofing, which is accepte

null 11 Nov 3, 2022