RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 332 and 375 in dimension 3 at /tmp/pip-req-build-rz55_vgo/aten/src/TH/generic/THTensor.cpp:612

Hi,I am working on Stage 3: Train Restoration Network in Train. Run into this difficulty，this problem how to solve?

Thanks for sharing your great works. It is an innotative step to construct GT correspondences through perspective transformations and enable the explicit supervision. I add some codes to test the performance on Sun80, Urban100 and Manga109。There are no more changes to the released code. The first line is the result reported in the paper. The second line is tested from the released checkpoint. In the third line, we retrain the model on Stage3. CUFED has significant gain but other datasets all drop.

where is the code of super-resolution Restoration Module？ Which file contains the super-resolution Restoration Module operation？ Can I use another network structure for super-resolution？ Looking forward to your reply

I was looking through ref_cufed_dataset.py. Saw under the dataloader that the code downsamples and upsamples both original and reference images:

        # downsample image using PIL bicubic kernel
        lq_h, lq_w = gt_h // scale, gt_w // scale
        
        img_in_lq = img_in_pil.resize((lq_w, lq_h), Image.BICUBIC)
        img_ref_lq = img_ref_pil.resize((lq_w, lq_h), Image.BICUBIC)
        
        # bicubic upsample LR
        img_in_up = img_in_lq.resize((gt_w, gt_h), Image.BICUBIC)
        img_ref_up = img_ref_lq.resize((gt_w, gt_h), Image.BICUBIC)

I assume we downsample and upsample the images to make them low-resolution, but why do it for the reference image as well? Shouldn't the reference image be high-resolution?

1. Can scale only be 4? Can it be adjusted to other scales? If so, what codes need to be modified and what should be paid attention to.

2. In the test phase, can I not sample(down) the input? I adjust the relevant code, but report an error...

I look forward to your reply!

In ContrasDataset, you resize the input. (opt['gt_size'] is 160 in training config.)

        gt_h, gt_w = self.opt['gt_size'], self.opt['gt_size']
        # in case that some images may not have the same shape as gt_size
        img_in = mmcv.imresize(img_in, (gt_w, gt_h), interpolation='bicubic')

However, in ContrasValDataset, there is not "resize".

        gt_h, gt_w, _ = img_in.shape

        H_inverse = self.transform_matrices[index]
        img_in_transformed = cv2.warpPerspective(
            src=img_in, M=H_inverse, dsize=(gt_w, gt_h))

Therefore, I have the question why you don't train the correspondence net from original HR (GT).

Hi, thanks for your interesting work. I cannot find any scheduler.step() call in your code, even though there are several scheduler setups (In training logs, lr is always constant during the training process). Is this a bug or is it intentional?

Hi there,

Thanks for your great work! Where did you set the model checkpoints are saved in the "experiments" folder? May you please direct me to that statement?

Thanks

In training stage1 and stage2, ContrasValDataset why does use low resolution images to generate ref images by resizing instead of reading ref images directly

    img_path  = self . paths [index ]['in_path' ]
    img_bytes  = self . file_client . get ( img_path ,  'in' ) 
    img_in  = mmcv . imfrombytes ( img_bytes ). astype ( np . float32 )  / 255.

    gt_h,  gt_w = self . opt ['gt_size' ],  self . opt ['gt_size' ]
    # in case that some images may not have the same shape as gt_size 
    img_in  = mmcv . imresize ( img_in , ( gt_w,  gt_h),  interpolation ='bicubic' ) 

    # augmentation: flip, rotation 
    img_in  = augment ([ img_in ],  self . opt ['use_flip' ],  self . opt ['use_rot' ]) 

    # image pair generation 
    img_in_transformed ,  H ,  H_inverse  = image_pair_generation ( 
        img_in , ( 0,  10),  160) 

 
    return  {
        'img_in':  img_in , 
        'img_in_up':  img_in_up , 
        'img_ref' :  img_in_transformed , 
        'img_ref_up' :  img_in_transformed_up , 
        'transformed_coordinate' :  transformed_coordinate 
    }

Excuse me, does the gt-size need to change according to the size of the ref image during the training? For example, if my ref image size is 720 * 1280, do I need to change the gt-size to 720? In addition, how many epochs are appropriate for stage1, stage2 and stage3 in the training

Excuse me, I want to train and test my own dataset. If the 180 * 320 super resolution to 720 * 1080, what size should the input and ref image be,thanks

I would like to ask the author if the c2_matching code supports distributed training, and if so, can you share the command，such as<python -m torch.distributed.launch --nproc_per_node=2 --launcher pytorch>，thank you！