Thank you for this great job. I want to know how i can evaluate the results of dancetrack? I successfully run this project on dancetrack test, but I can not get the final results like HOTA, MOTA etc.

Could you please provide the version of pretrain model. I downloaded the r50_deformable_detr-checkpoint.pth but the there is an error in loading state_dict for MOTR

Hi, I've read your paper in ECCV 2022, believe that your research is truly meaningful.

I'm trying to reproduce your experiment. At section 4.2 in your paper, you have mentioned that "provide a memory-optimized version that can be trained on NVIDIA 2080 Ti GPUs". But I didn't find any details in this repository.

Did you release the memory-optimized code? If not, will you release this part?

Thanks a lot for your contribution.

hi! when i run r50_motr_train.sh ,i meet some problems as follow:

(1)As for dataset ,according to FairMOT ,i put the MOT17 and crowdhuman in the right path ,but their labels I download have something different with the format of your requirements, the label_with_ids is empty , i don't kown how to generate their label. in the other word ,what code should i run to generate their label?

(2)As for model pretrain parameters, I have found coco_model_final.pth in Deformable DETR .But there are 5 models pth in Main Results ,i don't know which pth i need.

If anyone knows the answer, please to reply me and I'd really appreciate it.

Hi, I found that the performance of the model on multiclass dataset BDD100k is lower than using traditional detector-based trackers such as QDTrack, and I think this also happens with other transformer-based trackers such as GTR. From my experiments the mIDF1 could reach about 50 and IDF1 about 70 using QDTrack on BDD100k, but transformer-based trackers are not working so well. But I see on single-class tracking datasets like MOT17 transformer-based model works pretty well. Have you done any experiments about this and do you have any insights about the possible reasons for this? Please correct me if I am wrong, thank you!

Thanks for the impressive work! I think both the code and the paper are very interesting and insightful. However, I have a question about the track query and object query design. According to my understanding, when doing tracking, you concatenated the track queries and the object queries for existing and new born object detection and tracking. I am a little bit confused that how do you prevent the new born object queries detect the existing objects? To better describe, I just show a small example in this discussion.

For example, the frame_0 (initial frame) detected 2 objects and these 2 objects' corresponding query features are concatenated to the new born queries, which yields the frame_1 (next frame) object queries to be 302*256 where 302 is the total number of object queries. Since the first 300 object queries positions and features are random-initialized, is there a mechanism or module in the MOTR to prevent these 300 object queries not re-detect the 2 objects that you detected from the initial frame? Thank you so much!

Hi, I found that when I use 8 2080Ti to train this model. At the initial stage, the GPU memory occupation is 6/8, but the GPU memory is soon out. So do you have an explanation for this? And what's the suggested regime to train the MOTR model?

run: sh configs/r50_motr_submit.sh

result: .... filter init box 442 44 filter init box 443 44 save init track 443 45 save init track 444 44 save init track 449 46 totally 13 boxes are filtered. totally 4107 dts 0 occlusion dts ......

while not accuracy rate or error.

Hi, Thank you for this useful and very interesting work.

I have a question about the following;

1. How to view demo in video, to see detected and tracked object.
2. What is the total time used to train the model, and what is the speed (in frame per second ) during testing?
3. Which location to put dataset having labels and images_with_ids
4. Can you share the instruction on how to use the model on a custom dataset?

Thank you.

Hi,

Thanks for the wonderful work.

May I ask, compared to the implementation on MOT17 dataset, what adaptation you made for DanceTrack? I noticed the change of image resolution during training, etc. Just in case I miss anything. Thank you.

Thanks for the great work.

Looks like I can't find MOTR's results on the MOT17 leaderboard. Could you please provide the full evaluation results, including such as HOTA, AssA, Frag and so on, to provide more sense of its performance?

Moreover, have you tried MOTR on MOT20 as well?

I get the following error when trying to run the demo code:

Traceback (most recent call last):
  File "demo.py", line 284, in <module>
    detector.run()
  File "demo.py", line 254, in run
    res = self.model.inference_single_image(cur_img.cuda().float(), (self.dataloader.seq_h, self.dataloader.seq_w), track_instances)
  File "/root/miniconda3/envs/deformable_detr/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
    return func(*args, **kwargs)
  File "/h1/abhishek/blog/MOTR/models/motr.py", line 586, in inference_single_image
    track_instances=track_instances)
  File "/h1/abhishek/blog/MOTR/models/motr.py", line 515, in _forward_single_image
    hs, init_reference, inter_references, enc_outputs_class, enc_outputs_coord_unact = self.transformer(srcs, masks, pos, track_instances.query_pos, ref_pts=track_instances.ref_pts)
  File "/root/miniconda3/envs/deformable_detr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/h1/abhishek/blog/MOTR/models/deformable_transformer_plus.py", line 162, in forward
    memory = self.encoder(src_flatten, spatial_shapes, level_start_index, valid_ratios, lvl_pos_embed_flatten, mask_flatten)
  File "/root/miniconda3/envs/deformable_detr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/h1/abhishek/blog/MOTR/models/deformable_transformer_plus.py", line 266, in forward
    output = layer(output, pos, reference_points, spatial_shapes, level_start_index, padding_mask)
  File "/root/miniconda3/envs/deformable_detr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/h1/abhishek/blog/MOTR/models/deformable_transformer_plus.py", line 237, in forward
    src = self.forward_ffn(src)
  File "/h1/abhishek/blog/MOTR/models/deformable_transformer_plus.py", line 225, in forward_ffn
    src2 = self.linear2(self.dropout2(self.activation(self.linear1(src))))
  File "/root/miniconda3/envs/deformable_detr/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/miniconda3/envs/deformable_detr/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/root/miniconda3/envs/deformable_detr/lib/python3.7/site-packages/torch/nn/functional.py", line 1612, in linear
    output = input.matmul(weight.t())
RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

I have been running your demo in my docker with the following configuration :

FROM nvidia/cuda:9.2-cudnn7-devel-ubuntu18.04

RUN apt-get update
RUN apt-get install ffmpeg libsm6 libxext6  -y

Thanks for your work！ When this algorithm is applied to data sets with ego motion, how does it perform? How can it be improved to be deployed on mobile robots taking ego-motion into account?

Hi! Congratulations for your nice work. When I try to use the supported model to run eval.py, I get a very shocking results:

Then I use my trained model which runs 70+ epochs on crowdhuman and mot17 to eval and get the following result:

I am shocked by it's performance on the train dataset, so I wonder know is it normal?

Hi, thanks for sharing the codes, but I am confused about the bbox foramt as follows: in dataset/joint.py/class DetMOTDetection:--> """ def _pre_single_frame(self, idx: int):

**if osp.isfile(label_path): labels0 = np.loadtxt(label_path, dtype=np.float32).reshape(-1, 6)

        # normalized cewh to pixel xyxy format
        labels = labels0.copy()
        labels[:, 2] = w * (labels0[:, 2] - labels0[:, 4] / 2)
        labels[:, 3] = h * (labels0[:, 3] - labels0[:, 5] / 2)
        labels[:, 4] = w * (labels0[:, 2] + labels0[:, 4] / 2)
        labels[:, 5] = h * (labels0[:, 3] + labels0[:, 5] / 2)**

The dataset I am using is MOT17, it seems that the bboxes format that GT files provide are (x,y,w,h), but here in the above func, the gt boxes are used as (center_x,center_y,w,h), I am not sure if there is something wrong with it ?

Hi, is there any way to generate the output attention maps of model.transformer.decoder.layers[i].cross_attn layer? when I follow the referenced functions, I finally get stuck in MSDA.ms_deform_attn_forward function in the forward method of the MSDeformAttnFunction class which is located at ./models/ops/functions/ms_deform_attn_func.py file, and I couldn't find any argument to set True to get the attention map in output.

./models/deformable_transformer_plus/DeformableTransformerDecoderLayer

./models/ops/modules/ms_deform_attn.py

./models/ops/functions/ms_deform_attn_func.py