A new video text spotting framework with Transformer

Hi, the recognition model in your paper is MASTER. l know for some reasons u can't open the recognition code. Could you please tell me whether u use the Vanilla MASTER or the modified one？Thanks！

Hi,

Thanks for your great work!

I am a bit confused after I run the python track_tools/Evaluation_ICDAR15_video/vis_tracking.py Then, I get "No such file or directory: './output/ICDAR15/test/best_json_tracks/res_video_1.json'

I have seen issue #2 , and confirm I have run the python track_tools/convert_ICDAR15video_to_coco.py But, it seems that the "res_video_1.json" has not been generated successfully. I only find "train.json" and "test.json" under the "annotations_coco_rotate/", should I name one of them to "res_video_1.json" and copy it to "./output/ICDAR15/test/best_json_tracks/res_video_1.json"?

Plz, help me! Thanks a lot!

Traceback (most recent call last): File "main_track.py", line 363, in main(args) File "main_track.py", line 326, in main model, criterion, data_loader_train, optimizer, device, epoch, args.clip_max_norm) File "TransVTSpotter/engine_track.py", line 41, in train_one_epoch for _ in metric_logger.log_every(range(len(data_loader)), print_freq, header): File "TransVTSpotter/util/misc.py", line 260, in log_every meters=str(self), File "TransVTSpotter/util/misc.py", line 210, in str "{}: {}".format(name, str(meter)) File "TransVTSpotter/util/misc.py", line 109, in str median=self.median, File "TransVTSpotter/util/misc.py", line 88, in median return d.median().item() RuntimeError: median cannot be called with empty tensor

l think there might be something wrong with the datasets. My path of the datasets is as below:

Is that right? Can u give me some examples of the structure of the datasets or the solution to this error? Thanks!

there was an error "FileNotFoundError: [Errno 2] No such file or directory: './output/ICDAR15/test/best_json_tracks/res_video_1.mp4.json"

I downloaded the IC15 video dataset and run "python track_tools/convert_ICDAR15video_to_coco.py".

And I couldn't find files in json or jpg format downloaded from the icdar website https://rrc.cvc.uab.es/?ch=3&com=downloads. Unziped files only have '***.mp4' or '***.xml' and '***.txt'

How could I get the json annotatation files such as 'res_video_1.mp4.json'?

Thank you for the nice work! I'm having problems reproducing the results in your paper. I was hoping you can help.

I have done the following steps.

1. Download ICDAR15 video training and official test video dataset.
2. Prepare training and test dataset folder using: video2frames & convert_ICDAR15video_to_coco.
3. Download pretrain_coco.pth from your Baidu drive.
4. Train on ICDAR15 video using python -m torch.distributed.launch --nproc_per_node=8 --use_env main_track.py --output_dir ./output/icdar_tiv --dataset_file text --coco_path "${MY_DATA_DIR}/icdar_tiv" --batch_size 2 --with_box_refine --num_queries 300 --epochs 80 --lr_drop 40 --resume ./pths/pretrain_coco.pth.
5. Generate inferences using trained model on official test set: python main_track.py --eval --output_dir ./output/icdar_tiv_submit --resume ./output/icdar_tiv/checkpoint0079.pth --dataset_file text --coco_path "${MY_DATA_DIR}/icdar_tiv_test" --batch_size 1 --with_box_refine --num_queries 300
6. Zip up the results in output/icdar_tiv_submit/text/xml_dir.
7. Submit results to official ICDAR2015.

The resulting MOTA is 2.08% and very far from the expected ~45%. Note that the "Mostly Matched" is 842 matching reported results, so it seems that the object detection is working, but tracking is failing. Am I missing something from the code? Thanks for any help.