Hi! Thanks for your work! When I try train the original SRFBNx2 in my own dataset, the gradient explosion. The logging is this,

===> Training Epoch: [1/1000]... Learning Rate: 0.000100 Epoch: [1/1000]: 100%|███| 985/985 [02:24<00:00, 7.63it/s, Batch Loss: 18.7957]

Epoch: [1/1000] Avg Train Loss: 18.989669 ===> Validating... [MyImage] PSNR: 32.84 SSIM: 0.9147 Loss: 4.540801 Best PSNR: 32.84 in Epoch: [1] ===> Saving last checkpoint to [experiments/SRFBN_in3f32_x2/epochs/last_ckp.pth] ...] ===> Saving best checkpoint to [experiments/SRFBN_in3f32_x2/epochs/best_ckp.pth] ...]

===> Training Epoch: [2/1000]... Learning Rate: 0.000100 Epoch: [2/1000]: 54%|██████████████████████████▎ | 530/985 [01:18<01:07, 6.78it/s, Batch Loss: 3829046.5000][Warning] Skip this batch! (Loss: 11705110.0) Epoch: [2/1000]: 56%|██████████████████████████ | 547/985 [01:20<01:03, 6.84it/s, Batch Loss: 930627584.0000][Warning] Skip this batch! (Loss: 2895044864.0) Epoch: [2/1000]: 57%|█████████████████████████▋ | 562/985 [01:22<01:02, 6.80it/s, Batch Loss: 24168931328.0000]

I just change the dataset and set a bigger dataset repeat. By the way, my dataset's HR images is w//4 and h// ==0. And I train the FBSRNx4 successful, get the PSNR=29.18 in the dataset. What is the problem do you think? Thanks!

Hi, thanks for sharing your work. I'm using windows 10. I use the code of EDSR to train SRFBN on 2080Ti GPU. I always get 'OSError: [Errno 5] Input/output error' when training model on DIV2K and Flickr2K. and I find it would not happen when training model just on DIV2K.

Hello, I have a question that I need your help. When I reproduced the model of the paper, the best PSNR of Set5 was 32.13. Could you please tell me whether other skills are useful in training the model? Thx

I can't reproduce the average inference time that reported as 0.011s on the benchmark Urban100 with an Nvidia GTX1080Ti GPU. In my case, it needs about 0.4s (just inference time) to make a super resolution picture.

Thanks for your impressive work. You mentioned that you retrained VDSR and SRCNN for BD/DN model. Could you tell me that you strictly followed the original settings and retrained the VDSR, SRCNN model? or simply use same settings as your main methods but the structure?

Thanks for your great work。I trained on my own dataset for x8 scale，but when I try to test is，I got an error。 It looks like the data size in chechpoint is half in the model

Thanks your project, but when I use 291 dataset to train, I get error:

===> Training Epoch: [1/1000]... Learning Rate: 0.000100 Epoch: [1/1000]: 0%| | 0/114 [00:00<?, ?it/s] Traceback (most recent call last): File "train.py", line 131, in main() File "train.py", line 67, in main for iter, batch in enumerate(train_loader): File "C:\Users\h439\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 336, in next return self._process_next_batch(batch) File "C:\Users\h439\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 357, in _process_next_batch raise batch.exc_type(batch.exc_msg) ValueError: Traceback (most recent call last): File "C:\Users\h439\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 106, in _worker_loop samples = collate_fn([dataset[i] for i in batch_indices]) File "C:\Users\h439\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 106, in samples = collate_fn([dataset[i] for i in batch_indices]) File "E:\DYng_Z\3-image_super\code\SRFBN_CVPR19\data\LRHR_dataset.py", line 40, in getitem lr, hr = self._get_patch(lr, hr) File "E:\DYng_Z\3-image_super\code\SRFBN_CVPR19\data\LRHR_dataset.py", line 74, in _get_patch lr, hr, LR_size, self.scale) File "E:\DYng_Z\3-image_super\code\SRFBN_CVPR19\data\common.py", line 134, in get_patch ix = random.randrange(0, iw - ip + 1) File "C:\Users\h439\Anaconda3\lib\random.py", line 199, in randrange raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width)) ValueError: empty range for randrange() (0,-5, -5)

I have no idea how to solve it. Can you help me?

Hi， From the paper,I noticed the sentence “ We also observe that fine-tuning on a network pretrained on the BI degradation model leads to higher PSNR values than training from scratch. ”Could you tell me the details of "fine-tuning". Look forward to your reply! Thanks!

Hi, there. I have seen that the speed of your method is really fast compared to the other methods according to the last pages of your paper. Have you compared your method with SRCNN or FSRCNN, which claimed to be 40 times faster than SRCNN?

I want to quick run the test.py ,download pre-trained model and mv to models, add SRSolver.py 305:checkpoint = torch.load(model_path,map_location='cpu'), but errors :

File "/*/srfbn/solvers/SRSolver.py", line 308, in load else self.model.module.load_state_dict

AttributeError: 'SRFBN' object has no attribute 'module'

please give me some advices, thanks

Hey,

First, thank you very much for this amazing paper. In my case, I would like to train the model for other degradation methods than the ones presented in your work. If I have well understood, for complex degradation models the complexity of the HR ground truth increase over feedback iterations. However, I don't see any option to specify ground truth at feedback-block level. Is there any way to do this with your code ?

Thank you in advance and have a nice day, Charles

Hi, thank you for sharing for the code, I would like to ask if the final test is with the last epoch of model training, or the validation set best model for testing?

Hello, thank you for your dedicated work. However, when I try to re-implement your complex degradation, particularly BD degradation model, I receive very high psnr on x4 scale. I'm in around epoch 70 and psnr is around 32dB. I validate on set5 dataset with the same way I created training data on DIV2K, which is adding noise to HR (i used cv2.GaussianBlur(hr,(7,7),1.6) ), then apply bicubic downsampling to it to get LR images. I am so confused about this result and I hope to get your comment on this to know if I am wrong or not. Thank you.

Hi, @Paper99 I am try to reproduct RCAN based on your code. my code:

import torch
from torch import nn as nn
import math
# from basicsr.models.archs.arch_util import Upsample, make_layer

def make_layer(basic_block, num_basic_block, **kwarg):
    """Make layers by stacking the same blocks.
    Args:
        basic_block (nn.module): nn.module class for basic block.
        num_basic_block (int): number of blocks.
    Returns:
        nn.Sequential: Stacked blocks in nn.Sequential.
    """
    layers = []
    for _ in range(num_basic_block):
        layers.append(basic_block(**kwarg))
    return nn.Sequential(*layers)

class Upsample(nn.Sequential):
    """Upsample module.
    Args:
        scale (int): Scale factor. Supported scales: 2^n and 3.
        num_feat (int): Channel number of intermediate features.
    """

    def __init__(self, scale, num_feat):
        m = []
        if (scale & (scale - 1)) == 0:  # scale = 2^n
            for _ in range(int(math.log(scale, 2))):
                m.append(nn.Conv2d(num_feat, 4 * num_feat, 3, 1, 1))
                m.append(nn.PixelShuffle(2))
        elif scale == 3:
            m.append(nn.Conv2d(num_feat, 9 * num_feat, 3, 1, 1))
            m.append(nn.PixelShuffle(3))
        else:
            raise ValueError(f'scale {scale} is not supported. '
                             'Supported scales: 2^n and 3.')
        super(Upsample, self).__init__(*m)

class ChannelAttention(nn.Module):
    """Channel attention used in RCAN.
    Args:
        num_feat (int): Channel number of intermediate features.
        squeeze_factor (int): Channel squeeze factor. Default: 16.
    """

    def __init__(self, num_feat, squeeze_factor=16):
        super(ChannelAttention, self).__init__()
        self.attention = nn.Sequential(
            nn.AdaptiveAvgPool2d(1),
            nn.Conv2d(num_feat, num_feat // squeeze_factor, 1, padding=0),
            nn.ReLU(inplace=True),
            nn.Conv2d(num_feat // squeeze_factor, num_feat, 1, padding=0),
            nn.Sigmoid())

    def forward(self, x):
        y = self.attention(x)
        return x * y


class RCAB(nn.Module):
    """Residual Channel Attention Block (RCAB) used in RCAN.
    Args:
        num_feat (int): Channel number of intermediate features.
        squeeze_factor (int): Channel squeeze factor. Default: 16.
        res_scale (float): Scale the residual. Default: 1.
    """

    def __init__(self, num_feat, squeeze_factor=16, res_scale=1):
        super(RCAB, self).__init__()
        self.res_scale = res_scale

        self.rcab = nn.Sequential(
            nn.Conv2d(num_feat, num_feat, 3, 1, 1), nn.ReLU(True),
            nn.Conv2d(num_feat, num_feat, 3, 1, 1),
            ChannelAttention(num_feat, squeeze_factor))

    def forward(self, x):
        res = self.rcab(x) * self.res_scale
        return res + x


class ResidualGroup(nn.Module):
    """Residual Group of RCAB.
    Args:
        num_feat (int): Channel number of intermediate features.
        num_block (int): Block number in the body network.
        squeeze_factor (int): Channel squeeze factor. Default: 16.
        res_scale (float): Scale the residual. Default: 1.
    """

    def __init__(self, num_feat, num_block, squeeze_factor=16, res_scale=1):
        super(ResidualGroup, self).__init__()

        self.residual_group = make_layer(
            RCAB,
            num_block,
            num_feat=num_feat,
            squeeze_factor=squeeze_factor,
            res_scale=res_scale)
        self.conv = nn.Conv2d(num_feat, num_feat, 3, 1, 1)

    def forward(self, x):
        res = self.conv(self.residual_group(x))
        return res + x


class RCAN(nn.Module):
    """Residual Channel Attention Networks.
    Paper: Image Super-Resolution Using Very Deep Residual Channel Attention
        Networks
    Ref git repo: https://github.com/yulunzhang/RCAN.
    Args:
        num_in_ch (int): Channel number of inputs.
        num_out_ch (int): Channel number of outputs.
        num_feat (int): Channel number of intermediate features.
            Default: 64.
        num_group (int): Number of ResidualGroup. Default: 10.
        num_block (int): Number of RCAB in ResidualGroup. Default: 16.
        squeeze_factor (int): Channel squeeze factor. Default: 16.
        upscale (int): Upsampling factor. Support 2^n and 3.
            Default: 4.
        res_scale (float): Used to scale the residual in residual block.
            Default: 1.
        img_range (float): Image range. Default: 255.
        rgb_mean (tuple[float]): Image mean in RGB orders.
            Default: (0.4488, 0.4371, 0.4040), calculated from DIV2K dataset.
    """

    def __init__(self,
                 num_in_ch=3,
                 num_out_ch=3,
                 num_feat=64,
                 num_group=10,
                 num_block=16,
                 squeeze_factor=16,
                 upscale=2,
                 res_scale=1,
                 img_range=255.,
                 rgb_mean=(0.4488, 0.4371, 0.4040)):
        super(RCAN, self).__init__()

        self.img_range = img_range
        self.mean = torch.Tensor(rgb_mean).view(1, 3, 1, 1)

        self.conv_first = nn.Conv2d(num_in_ch, num_feat, 3, 1, 1)
        self.body = make_layer(
            ResidualGroup,
            num_group,
            num_feat=num_feat,
            num_block=num_block,
            squeeze_factor=squeeze_factor,
            res_scale=res_scale)
        self.conv_after_body = nn.Conv2d(num_feat, num_feat, 3, 1, 1)
        self.upsample = Upsample(upscale, num_feat)
        self.conv_last = nn.Conv2d(num_feat, num_out_ch, 3, 1, 1)

    def forward(self, x):
        # print(x.shape)
        self.mean = self.mean.type_as(x)

        x = (x - self.mean) * self.img_range
        x = self.conv_first(x)
        res = self.conv_after_body(self.body(x))
        res += x

        x = self.conv_last(self.upsample(res))
        x = x / self.img_range + self.mean
        # print(x.shape)
        # exit()
        return x

this code can run, but the loss is very high(about 1e30). I feel so confused about this, can you give me suggestions? my train.json:

{
    "mode": "sr",
    "use_cl": false,
    // "use_cl": true,
    "gpu_ids": [1],

    "scale": 2,
    "is_train": true,
    "use_chop": true,
    "rgb_range": 255,
    "self_ensemble": false,
    "save_image": false,

    "datasets": {
        "train": {
            "mode": "LRHR",
            "dataroot_HR": "/home/wangsen/ws/dataset/DIV2K/Augment/DIV2K_train_HR_aug/x2",
            "dataroot_LR": "/home/wangsen/ws/dataset/DIV2K/Augment/DIV2K_train_LR_aug/x2",
            "data_type": "npy",
            "n_workers": 8,
            "batch_size": 16,
            "LR_size": 48,
            "use_flip": true,
            "use_rot": true,
            "noise": "."
        },
        "val": {
            "mode": "LRHR",
            "dataroot_HR": "./results/HR/Set5/x2",
            "dataroot_LR": "./results/LR/LRBI/Set5/x2",
            "data_type": "img"
        }
    },

    "networks": {
        "which_model": "RCAN",
        "num_features": 64,
        "in_channels": 3,
        "out_channels": 3,
        "res_scale": 1,
        "num_resgroups":10, 
        "num_resblocks":20,
        "num_reduction":16
    },

    "solver": {
        "type": "ADAM",
        "learning_rate": 0.0002,
        "weight_decay": 0,
        "lr_scheme": "MultiStepLR",
        "lr_steps": [200, 400, 600, 800],
        "lr_gamma": 0.5,
        "loss_type": "l1",
        "manual_seed": 0,
        "num_epochs": 1000,
        "skip_threshold": 3,
        "split_batch": 1,
        "save_ckp_step": 100,
        "save_vis_step": 1,
        "pretrain": null,
        // "pretrain": "resume",
        "pretrained_path": "./experiments/RCAN_in3f64_x4/epochs/last_ckp.pth",
        "cl_weights": [1.0, 1.0, 1.0, 1.0]
    }
}

hi, i note that the size of input is changing according to the scale factor. such as scale is x2 ,the input patch size is 60x60,scale is x3,the input patch size is 50x50, scale is x4,the input patch size is 40x40. Why is it designed like this？the different of input patch size will affect the final result?