I have noticed that the memory requirements for the model change depending on whether the training starts from a freshly initialized model or a model initialized from a checkpoint.

I am training the model on NVidia RTX 2080Ti GPU, which provides 11GB of memory. In order to start the training without running into RuntimeError: CUDA error: out of memory exception, I need to set the number of video clips per batch equal to 3. More specifically, in terms of configuration settings:

SOLVER:
  VIDEO_CLIPS_PER_BATCH: 3

This produces the batch size equal to 6, since we have 2 random frames per clip, as given by the configuration below:

VIDEO:
  RANDOM_FRAMES_PER_CLIP: 2

However. if I restart the training from a previously stored checkpoint, the memory consumption decreases to such an extent, that I can add one more video clip per batch without crashing due to insufficient memory capacity. More concretely, my configuration allows the following:

SOLVER:
  VIDEO_CLIPS_PER_BATCH: 4

This does not seem to influence the model performance after training.

I have tried explicitly calling the garbage collector and emptying the CUDA cache using

import gc
import torch

gc.collect()
torch.cuda.empty_cache()

but to no avail.

My question is. What do you think might be causing this sort of memory leak? I have been working on this architecture for some time and yet I haven't found a reasonable explanation so far.

At this point, my pipeline involves two separate configurations. First, I run the training for 100 iterations, save the checkpoint, halt the training, and then restart it with a different configuration allowing a bigger batch size, and let it train as required. It is pretty cumbersome as well as highly unprofessional. I would like to understand the underlying cause.

Thank you for your input.

Thank you for your sharing. I follow the README.md instruction. when I input the following command, like this: python3 demos/demo.py --demo-video test.mp4 --track-class person --dump-video True

I can't see any generated result video in the file "demos/demos_vis/".

I have tried many commands, like these :

① python3 demos/demo.py --demo-video test.mp4 --track-class person --dump-video True --output-path demos/demos_vis

② python3 demos/demo.py --demo-video test.mp4 --track-class person --dump-video False

③ python3 demos/demo.py --demo-video test.mp4 --track-class person --dump-video False --output-path demos/demos_vis

and so on.

any suggestions for it ? How to output the result video ? Thank you

As per our capacity, we reduced it to 4 GPUs and kept the learning rate as default 0.02. After 40-60 iterations we started getting Nan value of losses We reduced the rate to 0.015 and then trained. Even with this for >200 iterations, sometimes it shows Nan loss values, sometimes it runs fine. Even after testing with Nan loss values, we found all the accuracy values in the output table came out to be Nan.

Thanks for the great works. when i train the model with MOT17 dataset by the following command:

python3 -m torch.distributed.launch --nproc_per_node=2 tools/train_net.py --config-file configs/dla/DLA_34_FPN_EMM_MOT17.yaml --train-dir my_train_results/MOT17_TEST/ --model-suffix pth

i got the error:

Traceback (most recent call last): File "tools/train_net.py", line 132, in main() File "tools/train_net.py", line 128, in main train(cfg, train_dir, args.local_rank, args.distributed, logger) File "tools/train_net.py", line 80, in train logger, tensorboard_writer File "./siammot/engine/trainer.py", line 51, in do_train result, loss_dict = model(images, targets) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/parallel/distributed.py", line 619, in forward output = self.module(*inputs[0], **kwargs[0]) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/apex-0.1-py3.6-linux-x86_64.egg/apex/amp/_initialize.py", line 197, in new_fwd **applier(kwargs, input_caster)) File "./siammot/modelling/rcnn.py", line 47, in forward features = self.backbone(images.tensors) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/modules/container.py", line 117, in forward input = module(input) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "./siammot/modelling/backbone/dla.py", line 297, in forward x5 = self.level5(x4) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "./siammot/modelling/backbone/dla.py", line 231, in forward x1 = self.tree1(x, residual) File "/home/sx/Documents/anaconda/anaconda3/envs/pt170/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, **kwargs) File "./siammot/modelling/backbone/dla.py", line 54, in forward out += residual RuntimeError: The size of tensor a (47) must match the size of tensor b (46) at non-singleton dimension 3 can anybody help me ? thank you !

This is my train.py parameter configuration: This is my data structure： But I have this problem： I think there's something wrong here.But I do not know how to solve, ask for help, thank you

In siam-mot/blob/main/siammot/data/adapters/augmentation/video_augmentation.py, line 99 the value getting assigned to "transform" seems to be a tuple but in line 105 "tranform(image)", transform is getting called like a function. Is there any error in our setting of data set or some error in code ?

I have a question rather than an issue. Why did you choose to use maskrcnn-benchmark, which is no longer actively supported? Do you think SiamMOT could use Detectron2 instead? If so, could you give me some tips on how to do this?

Hi, excellent work! But I have some questions. If I already have bboxes in the video and I only want to use the SiamMOT for tracking-by-detection, what should I do. Particularly, how to just inference with my bboxes. And, how to train SiamMOT with my bboxes and then inference. Thank you. (maybe issue #5 is the same problem with me)

Hi everyone, I am new to learning cv and deep learning, and I use colab as my machine. My input video is person_car.mp4, and the output video should be in demos/demo_vis, but when I run demo.py, it needs person_car.mp4 in demos/demo_vis too, that is strange. Can somebody explain? this is my github which store my colab file

Hello. I encountered this error when testing with the following command，“python3 -m tools.test_net --config-file configs/dla/DLA_34_FPN_EMM_MOT17.yaml --output-dir /home/yhb/下载/track_results --model-file /home/yhb/下载/DLA-34-FPN_EMM_crowdhuman_mot17 --test-dataset /media/yhb/Data/BaiduNetdiskDownload/MOT challenge/MOT17/test” error:test_net.py: error: unrecognized arguments: challenge/MOT17/test --test-path is the test path in MOT17 downloaded from the official website has not been modified. What changes do I need to make to --test-path? It is very helpful for beginners. Thank you very much!

Dear authors, Thanks for making your code available, I am grateful to you for it. In regards to the code base, I was wondering when would the weights for the MOTChallenge-2017 Test (Public detection) data set, as referenced in the model zoo readme be available?

Thanks again.

Bumps pillow from 9.0.1 to 9.3.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

I'm trying to run demos/demo.py but returns connection error after a while. I think that the model links are not working anymore

Downloading: "http://dl.yf.io/dla/models/imagenet/dla34-ba72cf86.pth" to C:\Users\xxxxx/.cache\torch\hub\checkpoints\dla34-ba72cf86.pth
Traceback (most recent call last):
  File "C:\Users\caros\anaconda3\lib\urllib\request.py", line 1346, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "C:\Users\caros\anaconda3\lib\http\client.py", line 1285, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "C:\Users\caros\anaconda3\lib\http\client.py", line 1331, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "C:\Users\caros\anaconda3\lib\http\client.py", line 1280, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "C:\Users\caros\anaconda3\lib\http\client.py", line 1040, in _send_output
    self.send(msg)
  File "C:\Users\caros\anaconda3\lib\http\client.py", line 980, in send
    self.connect()
  File "C:\Users\caros\anaconda3\lib\http\client.py", line 946, in connect
    self.sock = self._create_connection(
  File "C:\Users\caros\anaconda3\lib\socket.py", line 844, in create_connection
    raise err
  File "C:\Users\caros\anaconda3\lib\socket.py", line 832, in create_connection
    sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

I was trying to setup the demo locally but it does not work:

root@alex-OMEN:/opt/company/siam-mot# pip3 install -r requirements.txt
...
root@alex-OMEN:/opt/company/siam-mot# python3 demos/demo.py --demo-video ../long_video.mp4 --track-class  person_vehicel -dump-video True 
Traceback (most recent call last):
  File "demos/demo.py", line 5, in <module>
    from demos.demo_inference import DemoInference
ModuleNotFoundError: No module named 'demos'

Then I copied demo.py to the root of project and try again :

root@alex-OMEN:/opt/company/siam-mot# python3 demo.py --demo-video ../long_video.mp4 --track-class  person_vehicel -dump-video True 
Traceback (most recent call last):
  File "demo.py", line 5, in <module>
    from demos.demo_inference import DemoInference
  File "/opt/company/siam-mot/demos/demo_inference.py", line 10, in <module>
    from maskrcnn_benchmark.structures.bounding_box import BoxList
ModuleNotFoundError: No module named 'maskrcnn_benchmark'

I installed maskrcnn according to https://github.com/facebookresearch/maskrcnn-benchmark/blob/main/INSTALL.md

And the current error is :

root@alex-OMEN:/opt/company/siam-mot# python3 demo.py --demo-video ../long_video.mp4 --track-class  person_vehicel -dump-video True 
Traceback (most recent call last):
  File "demo.py", line 5, in <module>
    from demos.demo_inference import DemoInference
  File "/opt/company/siam-mot/demos/demo_inference.py", line 11, in <module>
    from maskrcnn_benchmark.utils.checkpoint import DetectronCheckpointer
  File "/usr/local/lib/python3.8/dist-packages/maskrcnn_benchmark-0.1-py3.8-linux-x86_64.egg/maskrcnn_benchmark/utils/checkpoint.py", line 7, in <module>
    from maskrcnn_benchmark.utils.model_serialization import load_state_dict
  File "/usr/local/lib/python3.8/dist-packages/maskrcnn_benchmark-0.1-py3.8-linux-x86_64.egg/maskrcnn_benchmark/utils/model_serialization.py", line 7, in <module>
    from maskrcnn_benchmark.utils.imports import import_file
  File "/usr/local/lib/python3.8/dist-packages/maskrcnn_benchmark-0.1-py3.8-linux-x86_64.egg/maskrcnn_benchmark/utils/imports.py", line 4, in <module>
    if torch._six.PY3:
AttributeError: module 'torch._six' has no attribute 'PY3'

What am I doing wrong? Is this project still actual?

I've running the code successfully,thank you for your great work! Now I want to do some research,for example, replacing the default spatial matching method with my newly proposed method. The question is: where can we modify the code in order to change the spatial matching method? I've read the related code (./siammot/modeling/track_head),but found nothing related to the matching part. Could you please tell me more about the spatial matching part? maybe I missed some information.