Hello! Thank you for this great work! I faced with small problem. I want to use 2 heads at the same time (to predict segmentation mask and 3D bbox). Now I can run it separately - or 3d bbox, or segmentation head. I am ready to take your trained weights and turn on 2 heads at the same time. But I don't understand where in code you turn on segmentation head? Please help.

best wishes, Ilya

I am a python novice, sorry to disturb everyone, as the question shows, how to run the following program in pycharm, I can only run successfully in cmdpython demo.py tracking,ddd --load_model ../models/nuScenes_3Dtracking.pth --dataset nuscenes --pre_hm --track_thresh 0.1 --demo ../videos/nuscenes_mini.mp4 --test_focal_length 633

Hi

Just one question about the features used here. It seems that, for the inference_prehm here, which stores the previous heatmaps that would be used for feature aggregation in the proposed MFW. The code used as follows:

https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/detector.py#L351

https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/detector.py#L354

And it looks like that the same heatmaps from the previous frame (t-1) are used twice. And by debug, it has been proved.

I personally think that, in TraDeS pipeline, you would like to use the features and corresponding heatmaps from (t-1) and (t-2), instead of repeated (t-1), for the aggregations of features. However, it looks that it is not.

Just kindly ask, are there some hidden tricks here? Many thanks.

Hi there, thank you for sharing your great work! I have a question here, please do correct me if I got it wrong:

We use a "class agnostic center heatmap"(of previous feat) P_agn times the previous feat, then passing it to the DCN along with the offset OD to get the "center attentive feature", which can be used to enhance the current feat. In your paper, it said P_agn was obtained from the heatmap output of CenterNet, but in the base_model section in your training code I found that the P_agn, aka 'pre_hm_i', was always generated directly from the ground truth.

base_model.py: trainer.py: generic_dataset.py, getitem: _get_pre_dets:

I also read your code for test, the Detector seemed ok since it used the previous heatmap result. Have I misunderstood anything here? Many thanks!

Thank you for your awesome research work. I have a question related to segmentation task by this repo. I followed your instructions to run validation on YoutubeVIS 2019, but the results is in tracking format. Does this repo support segmentation format? Thank you.

I have tried the demo (with default parameters) on a very simple video which is tracking only 1 person, and found 2 following problems

1. If it fails to detect the person at the current frame, it will assign a new track ID at the next frame
2. Sometimes there are 2 overlapped detection boxes (with different track IDs) with very high IOU at the same place How can I fine-tune the default parameters to overcome these problems?

Hi Could you please tell me the reason form in your code below: https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L679 https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L680 which are used for making the labels of cost volume. It seems that in the code: https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L677 https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/dataset/generic_dataset.py#L678 the labels for h-hm and w-hm as already made.

Hi Thanks for the work, really adimre that

There is a question about the code, in the follow: https://github.com/JialianW/TraDeS/blob/3eafd249ca0f18af8000d5798d4c552a0bd627ec/src/lib/utils/tracker.py#L196

Why you invalid the last embedding while measuring the similarity? I tried run the code without this, and the IDF1 drops (0.1), which means it is important.

Many thanks.

When I compiled DCNv2 with gcc 7.5.0, it crashed because of some function undeclared errors. Does anybody meets similar errors?

TraDeS-master/src/lib/model/networks/DCNv2/src/cpu/dcn_v2_cpu.cpp:224:9: error: ‘THFloatBlas_gemv’ was not declared in this scope
         THFloatBlas_gemv('t', k_, m_, 1.0f,
         ^~~~~~~~~~~~~~~~
TraDeS-master/src/lib/model/networks/DCNv2/src/cpu/dcn_v2_cpu.cpp:224:9: note: suggested alternative: ‘THFloatBlas_gemm’
         THFloatBlas_gemv('t', k_, m_, 1.0f,
         ^~~~~~~~~~~~~~~~
         THFloatBlas_gemm
error: command 'g++' failed with exit status 1

Thanks for your working.

We try to train on mot dataset. We don't know where Head Network(such as 2D detection, 3D detection and Instace segmentation) plays, can you tell me where this part is in the code.

Thanks a lot.

I have used sh experiments/mot17_test.sh to train my own model. And I can set the batch size. Because of the limitation of GPU performance, I changed it from 32 into 4. But I cannot see where to set the number of epoch. BTW, the notification said the program have to go through 140 epochs. I got the noti by running the command.

Hi, thanks for your works. When reading your paper, i have few questions about the baseline:

1. Baseline is build based on CenterNet with an extra head predicting tracking offset map O_B. As mention in #36 , backbone network just take a single image (frame at timestamp t) as input instead of previous_frame + current_frame + previous_detection_heatmap like CenterTrack? So I wonder how to predict offset map O_B with just one frame. My initial guess is that feature map of frame t and t-T is calculated independently, but attaching offset-map head on an aggregated feature map (e.g simple stack/average/subtraction) of t-T and t?
2. In Fig. 4, since Baseline doesn't have O_C (w/o CVA), does it means Baseline == Baseline + CVA or O_C == O_B in the baseline context?

Hello @JialianW I test the performance of the model you provided (nuscenes.pth)on the nuscenes validation set following the readme. But I got low performance which is different from the results in the paper.

### Final results ###

Per-class results:
		AMOTA	AMOTP	RECALL	MOTAR	GT	MOTA	MOTP	MT	ML	FAF	TP	FP	FN	IDS	FRAG	TID	LGD
bicycle 	0.000	1.495	0.552	0.000	1993	0.000	0.793	27	38	683.2	864	35219	893	236	90	1.07	1.82
bus     	0.062	1.420	0.224	0.394	2112	0.088	0.811	9	87	17.5	470	285	1638	4	19	0.70	4.02
car     	0.088	0.893	0.114	0.520	58317	0.059	0.486	133	3379	55.1	6582	3158	51646	89	166	2.04	4.35
motorcy 	0.000	1.579	0.545	0.000	1977	0.000	0.851	29	24	494.5	819	22430	899	259	118	1.14	2.32
pedestr 	0.000	1.394	0.639	0.000	25423	0.000	0.873	486	198	2454.6	10887	143963	9168	5368	1778	0.60	1.73
trailer 	0.000	1.614	0.543	0.000	2425	0.000	1.084	26	29	2907.8	869	151552	1108	448	169	1.32	2.16
truck   	0.003	1.341	0.093	0.139	9650	0.013	0.939	23	509	21.2	884	761	8756	10	30	0.70	3.08

Aggregated results:
AMOTA	0.022
AMOTP	1.391
RECALL	0.387
MOTAR	0.150
GT	14556
MOTA	0.023
MOTP	0.834
MT	733
ML	4264
FAF	947.7
TP	21375
FP	357368
FN	74108
IDS	6414
FRAG	2370
TID	1.08
LGD	2.78
Eval time: 3056.8s 

I also tested the results from CenterTrack and got good performance. So I ruled out the issues of data preparation and nuScenes dataset API. What anything else should I do to get the same performance as you provided? Thank you.

Thanks for your working. I test it on dataset and get a "results.json".But the competition needs different json style just Like { "video_id" : int, "category_id" : int, "segmentations" : [RLE or [polygon] or None], "score" : float, } How can I convet it?Thanks a lot.

Hello,many thanks for your work. As you can see from the following two pictures ，when person2, person6 and person7 overlap, their serial numbers are exchanged. Is there any way to solve this problem? Thanks again！