Trying your code in Colab, everything goes well for a few minutes (~300 epochs) and then it gets a CUDA memory error. Is there a leak somewhere? https://colab.research.google.com/drive/17UgzBmKtqRXniuG6fHqMGh3xIaVIJnd2?usp=sharing

Traceback (most recent call last):
  File "train.py", line 87, in <module>
    train_model(dataroot)
  File "train.py", line 58, in train_model
    losses = criterion(outputs, inputs)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/content/Splice/util/losses.py", line 64, in forward
    losses['loss_global_cls'] = self.calculate_crop_cls_loss(outputs['x_global'], inputs['B_global'])
  File "/content/Splice/util/losses.py", line 90, in calculate_crop_cls_loss
    cls_token = self.extractor.get_feature_from_input(a)[-1][0, 0, :]
  File "/content/Splice/models/extractor.py", line 84, in get_feature_from_input
    self.model(input_img)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/root/.cache/torch/hub/facebookresearch_dino_main/vision_transformer.py", line 212, in forward
    x = blk(x)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1071, in _call_impl
    result = forward_call(*input, **kwargs)
  File "/root/.cache/torch/hub/facebookresearch_dino_main/vision_transformer.py", line 108, in forward
    y, attn = self.attn(self.norm1(x))
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1071, in _call_impl
    result = forward_call(*input, **kwargs)
  File "/root/.cache/torch/hub/facebookresearch_dino_main/vision_transformer.py", line 85, in forward
    attn = (q @ k.transpose(-2, -1)) * self.scale
RuntimeError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 11.17 GiB total capacity; 10.35 GiB already allocated; 23.81 MiB free; 10.68 GiB reserved in total by PyTorch)

I'm trying to recreate the results in figure 9 in your paper. Specifically, disabling L_structure. Are both lambda_global_ssim and lambda_entire_ssim taken together form the entirety of L_struct or am I misreading the code?

So far tinkering with the various lambda terms, it seems the result always retains very high structural fidelity to the structure image.

Hello, very cool project! Is there a parameter to change the batch size? It seems that 8GB of VRAM isn't enough and I'm getting CUDA OOM errors. I've tried lowering the image resolutions but I still get the same issue.

Hi!

I love your work and have a question regarding multiple target images. Have you experimented with multiple target images? For example: Source Domain - Horses Target Domain - Zebras What happens if we pick a new zebra image as the target for every epoch?

Thanks!

Hey,

I noticed that in the calculation of self-similarity you calculate the norm of the same thing twice. I changed it, so it would be computed only once.

By emptying the hook handler list, python can remove the hook handlers that are no longer active (but still take up memory). In my test run, the loss keeps decreasing after this change

Pytorch version 1.10.1 CUDA version 11.3

When running the model, the time per iteration increases with each epoch. When printing import os, psutil; process = psutil.Process(os.getpid()); print(process.memory_info().rss), the memory used by the CPU increases with each epoch. After digging a bit, I think the issue is with the handlers, that are removed but aren't deleted from the hook_handlers list (the length of the hook_handlers list keeps increasing), so python's garbage collector does not collect them.

Hi, thanks for your great work. when I try to training model, meet this error. self.model = torch.hub.load('facebookresearch/dino:main', model_name).to(device) -- > URLError: It maybe my machine is no internet, but I try using "torch.save()" , and "torch.load" to replace "torch.hub.load " , it was error again. what shoud I fix this? Thanks again , waiting for your reply.

In the beginning, the seed is being created as int32 for some reason. Maybe pick a lower seed?

Traceback (most recent call last):
  File "train.py", line 88, in <module>
    train_model(dataroot)
  File "train.py", line 28, in train_model
    seed = np.random.randint(2 ** 32)
  File "mtrand.pyx", line 745, in numpy.random.mtrand.RandomState.randint
  File "_bounded_integers.pyx", line 1343, in numpy.random._bounded_integers._rand_int32
ValueError: high is out of bounds for int32