[CVPR 2022 Oral] Crafting Better Contrastive Views for Siamese Representation Learning

ch0 = max(int(height * h0) - h//2, 0) ch1 = min(int(height * h1) - h//2, height - h) In your implementation, ch0 can be larger than ch1. For example, ch0 and ch1 are both near 1.0. I think this situation may cause some errors.

Thanks for released the brilliant work again! When i was in traing process, i found that if i resume the training, the process will rsume from the saved checkpoint, but the self.use_box will return to the False, i mean the type of crop will becam to random crop, i was confused whether that will bother the end of training? or should i set the warmup_epochs to zero, when i resume trainging, in order to make sure that the type of crop is same as previously? Thanks for your reply.

hello, i've tried use your work on my dataset, and it made work, too.But when i try to enlarge my dataset, there is a problem occured: when the model train to 200 or 300epochs, the box update always failed, i try to figure out the reason, found that when the process going to update, the heat_map tensor is empty, i was confused how to solve this problem and does this occured by my dataset size? because when i use small dataset, this problem dosent happend.

Thank you for an interesting paper and easy to understand. Can I ask some questions?

1/ I still don't understand what is the class score you mentioned in section 3.4. Can you explain more? I checked in the code but I couldn't find it. Please correct me if I missed it.

2/ It's interesting that the learning rate for training the linear classifier is 10. Do you have any findings on this? or it's a heuristic configuration?

3/ What is the red plot in Section 4.4. Ablation Studies / Semantic-aware Localization

We also make comparison with RandomCrop that does not use localization (i.e., k = 0), and ground truth bounding boxes (the red plot).

Is it another experiment but has been removed in Fig 6.a ?

Thank you

Hi, great work! Thank you for sharing. In the codes, training data is loaded after the contrastive crop and then is used for training through all epochs. And for every location interval epoch, the box is updated. The box is used for the contrastive crop in the data loader process to generate better sample pairs. So how is the updating of the box used for generating better sample pairs?
I guess maybe we should load the training data every time we update the box, what do you think?

Hi author, comparative cropping is a very good idea. We used the last layer feature map or attention map in a similar project in moco v3 to make cropped rectangular borders according to a threshold of 0.1, but both had poor results. We updated the bounding box at 20 intervals and printed the information of the 300 epoch rectangular border to find h_min=0,w_min=0,h_max=0.9955,w_max=0.9955. This indicates that all pixel points in the produced heat map are larger than the threshold and therefore do not play the effect of comparative cropping. Please, are we supposed to enlarge the threshold? In addition, we observe that only the left and lower boundaries of the rectangular borders are related to the heat map, but the upper and right borders are uniformly sampled and computed according to the given scale and ratio. Then, if only the left and bottom borders of the rectangular border are considered, and the right and top borders are randomly selected can we guarantee that the main objects in the image are in the bounding box. This is also the point that makes us wonder. We hope you can give us an answer to the above question, thank you very much.

Thanks for sharing your great job! when i finish the pretext task in my own dataset, i got a checkpoint file, i check the keys in that file, it conbines statedict and boxes, i was confused about the box information is useful in the downstream task? or should i also consider to load the box weight?

Hi, Thanks for your amazing work.

I got the following error when i trying to run the linear classification experiment of SimCLR:

Command:

python DDP_linear.py configs/linear/cifar10_res18.py --load ./checkpoints/small/cifar10/simclr_alpha0.1_th0.1/last.pth

Error:

  File "/home/futong/jiarunliu/ContrastiveCrop/DDP_linear.py", line 276, in main_worker
    load_weights(cfg.load, model, optimizer, resume=False)
  File "/home/futong/jiarunliu/ContrastiveCrop/DDP_linear.py", line 107, in load_weights
    for k, v in state_dict.items():
UnboundLocalError: local variable 'state_dict' referenced before assignment

And I have checked the aviaible keys in SimCLR's checkpoint:

>>> ckpt = torch.load("checkpoints/small/cifar10/simclr_alpha0.1_th0.1/last.pth", map_location='cpu')
>>> ckpt.keys()
dict_keys(['optimizer_state', 'simclr_state', 'boxes', 'epoch'])

Maybe you should consider change the key here? https://github.com/xyupeng/ContrastiveCrop/blob/dab4100839972ecab0c864256d397867260e76a8/DDP_linear.py#L84-L85

In the ContrastiveCrop.py file: Line 44 to 47:

ch0 = max(int(height * h0) - h//2, 0)
ch1 = min(int(height * h1) - h//2, height - h)
cw0 = max(int(width * w0) - w//2, 0)
cw1 = min(int(width * w1) - w//2, width - w)

the upper bounds ch1 and cw1 may be wrong? It should be the addition instead of the subtraction? From my understanding, it ch1 and cw1 give the upper bounds (the maximum of the cropping region). Or I misunderstand the cropping candidate regions.

···
ch1 = min(int(height * h1) + h//2, height - h)
···
cw1 = min(int(width * w1) + w//2, width - w)

Thanks in advance.

You wrote that alpha was set to 0.6 for your method in sec.4.2, while I saw that alpha was set to 0.1 in the config you are given on the github in the folder "small". So I'm wondering which one is used for the results in paper?