Winning Solution in NTIRE19 Challenges on Video Restoration and Enhancement (CVPR19 Workshops) - Video Restoration with Enhanced Deformable Convolutional Networks. EDVR has been merged into BasicSR and this repo is a mirror of BasicSR.

Last update: Dec 25, 2022

Related tags

Overview

EDVR has been merged into BasicSR. This GitHub repo is a mirror of BasicSR. Recommend to use BasicSR, and open issues, pull requests, etc in BasicSR.

Note that this version is not compatible with previous versions. If you want to use previous ones, please refer to the old_version branch.

🚀 BasicSR

English | 简体中文 GitHub | Gitee码云

Google Colab: GitHub Link | Google Drive Link
Ⓜ️ Model Zoo ⏬ Google Drive: Pretrained Models | Reproduced Experiments ⏬ 百度网盘: 预训练模型 | 复现实验
📁 Datasets ⏬ Google Drive ⏬ 百度网盘 (提取码:basr)
📈 Training curves in wandb
💻 Commands for training and testing
⚡ HOWTOs

BasicSR (Basic Super Restoration) is an open source image and video restoration toolbox based on PyTorch, such as super-resolution, denoise, deblurring, JPEG artifacts removal, etc.
_{(ESRGAN, EDVR, DNI, SFTGAN)} _{(HandyView, HandyFigure, HandyCrawler, HandyWriting)}

✨ New Features

Nov 29, 2020. Add ESRGAN and DFDNet colab demo.
Sep 8, 2020. Add blind face restoration inference codes: DFDNet.
Aug 27, 2020. Add StyleGAN2 training and testing codes: StyleGAN2.

Sep 8, 2020. Add blind face restoration inference codes: DFDNet.
ECCV20: Blind Face Restoration via Deep Multi-scale Component Dictionaries
Xiaoming Li, Chaofeng Chen, Shangchen Zhou, Xianhui Lin, Wangmeng Zuo and Lei Zhang
Aug 27, 2020. Add StyleGAN2 training and testing codes.
CVPR20: Analyzing and Improving the Image Quality of StyleGAN
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen and Timo Aila
Aug 19, 2020. A brand-new BasicSR v1.0.0 online.

⚡ HOWTOs

We provides simple pipelines to train/test/inference models for quick start. These pipelines/commands cannot cover all the cases and more details are in the following sections.

GAN
StyleGAN2	Train	Inference
Face Restoration
DFDNet	-	Inference
Super Resolution
ESRGAN	TODO	TODO	SRGAN	TODO	TODO
EDSR	TODO	TODO	SRResNet	TODO	TODO
RCAN	TODO	TODO
EDVR	TODO	TODO	DUF	-	TODO
BasicVSR	TODO	TODO	TOF	-	TODO
Deblurring
DeblurGANv2	-	TODO
Denoise
RIDNet	-	TODO	CBDNet	-	TODO

🔧 Dependencies and Installation

Python >= 3.7 (Recommend to use Anaconda or Miniconda)
PyTorch >= 1.3
NVIDIA GPU + CUDA

Clone repo

git clone https://github.com/xinntao/BasicSR.git

Install dependent packages

cd BasicSR
pip install -r requirements.txt

Install BasicSR

Please run the following commands in the BasicSR root path to install BasicSR:
(Make sure that your GCC version: gcc >= 5)
If you do not need the cuda extensions:
dcn for EDVR
upfirdn2d and fused_act for StyleGAN2
please add --no_cuda_ext when installing
```
python setup.py develop --no_cuda_ext
```
If you use the EDVR and StyleGAN2 model, the above cuda extensions are necessary.
```
python setup.py develop
```
You may also want to specify the CUDA paths:
```
CUDA_HOME=/usr/local/cuda \
CUDNN_INCLUDE_DIR=/usr/local/cuda \
CUDNN_LIB_DIR=/usr/local/cuda \
python setup.py develop
```

Note that BasicSR is only tested in Ubuntu, and may be not suitable for Windows. You may try Windows WSL with CUDA supports :-) (It is now only available for insider build with Fast ring).

⏳ TODO List

Please see project boards.

🐢 Dataset Preparation

Please refer to DatasetPreparation.md for more details.
The descriptions of currently supported datasets (torch.utils.data.Dataset classes) are in Datasets.md.

💻 Train and Test

Training and testing commands: Please see TrainTest.md for the basic usage.
Options/Configs: Please refer to Config.md.
Logging: Please refer to Logging.md.

🏰 Model Zoo and Baselines

The descriptions of currently supported models are in Models.md.
Pre-trained models and log examples are available in ModelZoo.md.
We also provide training curves in wandb:

📝 Codebase Designs and Conventions

Please see DesignConvention.md for the designs and conventions of the BasicSR codebase.
The figure below shows the overall framework. More descriptions for each component:
Datasets.md | Models.md | Config.md | Logging.md

📜 License and Acknowledgement

This project is released under the Apache 2.0 license.
More details about license and acknowledgement are in LICENSE.

🌏 Citations

If BasicSR helps your research or work, please consider citing BasicSR.
The following is a BibTeX reference. The BibTeX entry requires the url LaTeX package.

@misc{wang2020basicsr,
  author =       {Xintao Wang and Ke Yu and Kelvin C.K. Chan and
                  Chao Dong and Chen Change Loy},
  title =        {BasicSR},
  howpublished = {\url{https://github.com/xinntao/BasicSR}},
  year =         {2020}
}

Xintao Wang, Ke Yu, Kelvin C.K. Chan, Chao Dong and Chen Change Loy. BasicSR. https://github.com/xinntao/BasicSR, 2020.

📧 Contact

If you have any question, please email [email protected].

Comments

not convergence, why?

19-07-01 09:44:55.725 - INFO: <epoch:221, iter: 448,100, lr:(2.583e-07,)>l_pix: 8.0168e+04 19-07-01 09:45:46.968 - INFO: <epoch:221, iter: 448,200, lr:(2.421e-07,)>l_pix: 6.3542e+04 19-07-01 09:46:39.644 - INFO: <epoch:222, iter: 448,300, lr:(2.267e-07,)>l_pix: 6.8591e+04 19-07-01 09:47:30.610 - INFO: <epoch:222, iter: 448,400, lr:(2.123e-07,)>l_pix: 6.2289e+04 19-07-01 09:48:21.469 - INFO: <epoch:222, iter: 448,500, lr:(1.987e-07,)>l_pix: 6.7909e+04 19-07-01 09:49:12.473 - INFO: <epoch:222, iter: 448,600, lr:(1.859e-07,)>l_pix: 5.4850e+04 19-07-01 09:50:03.302 - INFO: <epoch:222, iter: 448,700, lr:(1.741e-07,)>l_pix: 7.6995e+04 19-07-01 09:50:54.809 - INFO: <epoch:222, iter: 448,800, lr:(1.631e-07,)>l_pix: 6.6559e+04 19-07-01 09:51:45.599 - INFO: <epoch:222, iter: 448,900, lr:(1.531e-07,)>l_pix: 5.5888e+04 19-07-01 09:52:36.882 - INFO: <epoch:222, iter: 449,000, lr:(1.439e-07,)>l_pix: 5.7516e+04 19-07-01 09:53:27.801 - INFO: <epoch:222, iter: 449,100, lr:(1.355e-07,)>l_pix: 6.3166e+04 19-07-01 09:54:18.680 - INFO: <epoch:222, iter: 449,200, lr:(1.281e-07,)>l_pix: 5.9250e+04 19-07-01 09:55:09.492 - INFO: <epoch:222, iter: 449,300, lr:(1.215e-07,)>l_pix: 7.9755e+04 19-07-01 09:56:00.916 - INFO: <epoch:222, iter: 449,400, lr:(1.158e-07,)>l_pix: 7.5331e+04 19-07-01 09:56:51.746 - INFO: <epoch:222, iter: 449,500, lr:(1.110e-07,)>l_pix: 6.1268e+04 19-07-01 09:57:42.739 - INFO: <epoch:222, iter: 449,600, lr:(1.070e-07,)>l_pix: 5.8020e+04 19-07-01 09:58:34.047 - INFO: <epoch:222, iter: 449,700, lr:(1.039e-07,)>l_pix: 7.0684e+04 19-07-01 09:59:25.079 - INFO: <epoch:222, iter: 449,800, lr:(1.018e-07,)>l_pix: 6.1002e+04 19-07-01 10:00:16.479 - INFO: <epoch:222, iter: 449,900, lr:(1.004e-07,)>l_pix: 6.2216e+04 19-07-01 10:01:07.545 - INFO: <epoch:222, iter: 450,000, lr:(4.000e-04,)>l_pix: 6.2134e+04 19-07-01 10:01:07.546 - INFO: Saving models and training states. 19-07-01 10:01:58.558 - INFO: <epoch:222, iter: 450,100, lr:(4.000e-04,)>l_pix: 6.8700e+04 19-07-01 10:02:49.626 - INFO: <epoch:222, iter: 450,200, lr:(4.000e-04,)>l_pix: 5.7384e+04 19-07-01 10:03:42.350 - INFO: <epoch:223, iter: 450,300, lr:(4.000e-04,)>l_pix: 5.6992e+04 19-07-01 10:04:33.524 - INFO: <epoch:223, iter: 450,400, lr:(4.000e-04,)>l_pix: 5.5664e+04 19-07-01 10:05:24.229 - INFO: <epoch:223, iter: 450,500, lr:(4.000e-04,)>l_pix: 7.6057e+04 19-07-01 10:06:15.174 - INFO: <epoch:223, iter: 450,600, lr:(4.000e-04,)>l_pix: 7.2010e+04 19-07-01 10:07:06.118 - INFO: <epoch:223, iter: 450,700, lr:(4.000e-04,)>l_pix: 6.0641e+04 19-07-01 10:07:56.997 - INFO: <epoch:223, iter: 450,800, lr:(4.000e-04,)>l_pix: 5.9802e+04 19-07-01 10:08:47.783 - INFO: <epoch:223, iter: 450,900, lr:(4.000e-04,)>l_pix: 6.6285e+04 19-07-01 10:09:38.584 - INFO: <epoch:223, iter: 451,000, lr:(4.000e-04,)>l_pix: 6.3312e+04 19-07-01 10:10:29.622 - INFO: <epoch:223, iter: 451,100, lr:(3.999e-04,)>l_pix: 6.5559e+04 19-07-01 10:11:20.408 - INFO: <epoch:223, iter: 451,200, lr:(3.999e-04,)>l_pix: 7.3202e+04 19-07-01 10:12:11.144 - INFO: <epoch:223, iter: 451,300, lr:(3.999e-04,)>l_pix: 7.1605e+04 19-07-01 10:13:02.022 - INFO: <epoch:223, iter: 451,400, lr:(3.999e-04,)>l_pix: 6.4014e+04 19-07-01 10:13:53.078 - INFO: <epoch:223, iter: 451,500, lr:(3.999e-04,)>l_pix: 6.2185e+04 19-07-01 10:14:44.101 - INFO: <epoch:223, iter: 451,600, lr:(3.999e-04,)>l_pix: 5.7227e+04 19-07-01 10:15:34.804 - INFO: <epoch:223, iter: 451,700, lr:(3.999e-04,)>l_pix: 7.6518e+04 19-07-01 10:16:25.789 - INFO: <epoch:223, iter: 451,800, lr:(3.999e-04,)>l_pix: 6.0232e+04 19-07-01 10:17:16.691 - INFO: <epoch:223, iter: 451,900, lr:(3.998e-04,)>l_pix: 7.3397e+04

opened by robotzheng 14
about the loss function

Hi, when I trained with the CharbonnierLoss , the loss is very very big, but when I trained with L1 loss, it is normal, what caused this phenomenon, could you give me some advice?

opened by guiji0812 10
询问网络细节

很棒的工作！看完模型代码和论文，有几个问题。 1、Relu层的选择，是否有什么经验？比如，大多数SR网络用Relu，DBPN/RBPN用PRelu，ESRGAN和EDVR用LRelu，是基于什么考究？或者说是否以后的实验我什么都别管，直接用LRelu就是最优的？ 2、Resblock_noBN中，对Conv1Conv2使用kaiming初始化带来的提升我记得在ESRGAN的补充材料里也出现过，是否也是以后直接用就行，无计算成本的性能提升？ 3、如果要拓展参数量，一个是channel的变化，这个很简单，因为所有块都对齐了nf的；如果是拓展block数量呢？EDVR的输入参数，前后RB是可变的，但是PCD和AttentionFusion module，block是固定写死的，是否意味着，如果靠拓展深度加大网络参数量和计算量从而提升指标，A、这2个模块的深度不用管，加大深度不划算？B、如果这2个模块也要加大深度，应该加大哪些层的深度？C、第三个选择，只加大backRB的深度，frontRB和2个module的深度不用管？ Table4对照实验中的配置，和比赛最终结果的配置，是否只有channel和fusion后RB的深度变化了？ 4、相比SISR中常用的self-ensemble，8倍的计算量，EDVR的测试脚本中只有4倍，少了90度旋转的过程，这个是基于什么考究？是否因为提升几乎很小，所以就不用再通过2x计算量来提升指标了？ 5、Table5中4个track均以碾压的优势取得第一，这个非常值得肯定。第二名的方法在论文《Adapting Image Super-Resolution State-of-the-arts and Learning Multi-model Ensemble for Video Super-Resolution》中（arxiv上可以搜到），idea非常simple，是典型的刷榜流做法，第一个idea，把SISR（RCAN、RDN）中的输入改为相邻帧输入，直接改第一个Conv的输入channel，就已经提升了1dB，超过DUF，第二个idea就是3模型结果ensemble，结果进一步提升0.12（作用很小了，可能还不如self-ensemble带来的提升，不划算，这个先不提）。这里想看下DUF、标配RDN、RCAN和EDVR比赛配置的GFLOPS对比 6、预先进行拉伸在SISR中一直被视为不太好的做法，EDVR中用了bilinear拉伸LR_center和HR做残差，这个有实验过去掉这步吗？收敛速度上我觉得做拉伸做残差更快，但是精度不一定。（这个的目的是不是和EDSR-Pytorch中”add_mean”和”sub_mean”差不多的？） 7、训练细节上，本文使用的是Charbonnier Loss，这个最早我记得是在LapSRN中出现，之后普遍用L1Loss。这个是有什么考究吗？训练浅层网络作为深层网络的初始化，这个求具体细节，是哪些层的权重初始化了，深浅改变的是什么层的深度？ 8、有关PCD对齐模块（1）108、117行forward中，offset_conv2里concat乘2是什么作用？（后面TSA层好像也有乘2的）（2）为什么不排除中间帧(t=t+i)只做4个？不可以直接采用l1_fea的输入吗？

谢谢！

opened by splinter21 9
ImportError: cannot import name 'deform_conv_cuda'

When I run the test_Vid4_REDS4_with_GT.py, I got the importError.

`Traceback (most recent call last): File "/home/EDVR/codes/models/archs/EDVR_arch.py", line 8, in from models.archs.dcn.deform_conv import ModulatedDeformConvPack as DCN File "/home/EDVR/codes/models/archs/dcn/init.py", line 1, in from .deform_conv import (DeformConv, DeformConvPack, ModulatedDeformConv, ModulatedDeformConvPack, File "/home/EDVR/codes/models/archs/dcn/deform_conv.py", line 10, in from . import deform_conv_cuda ImportError: cannot import name 'deform_conv_cuda'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/EDVR/codes/test_EDVR_REDS_SR_L.py", line 16, in import models.archs.EDVR_arch as EDVR_arch File "/home/EDVR/codes/models/archs/EDVR_arch.py", line 10, in raise ImportError('Failed to import DCNv2 module.') ImportError: Failed to import DCNv2 module.`

My configuration as below: python 3.5 Pytorch 1.0 GCC 5.4.0 CUDA 8.0

what should I do to handle the error? thanks in advance.

opened by dongxijia 8
How about this warning information?

19-07-14 21:35:36.666 - WARNING: Offset mean is 9624.6220703125, larger than 100. 19-07-14 21:35:36.669 - WARNING: Offset mean is 65085.6953125, larger than 100. 19-07-14 21:35:36.674 - WARNING: Offset mean is 189.6161651611328, larger than 100. 19-07-14 21:35:36.675 - WARNING: Offset mean is 9624.8037109375, larger than 100. 19-07-14 21:35:36.678 - WARNING: Offset mean is 65086.70703125, larger than 100. 19-07-14 21:35:39.024 - WARNING: Offset mean is 168.29945373535156, larger than 100. 19-07-14 21:35:39.026 - WARNING: Offset mean is 8571.626953125, larger than 100. 19-07-14 21:35:39.029 - WARNING: Offset mean is 57954.5703125, larger than 100. 19-07-14 21:35:39.034 - WARNING: Offset mean is 168.32696533203125, larger than 100. 19-07-14 21:35:39.036 - WARNING: Offset mean is 8572.4208984375, larger than 100. 19-07-14 21:35:39.039 - WARNING: Offset mean is 57959.84375, larger than 100. 19-07-14 21:35:39.044 - WARNING: Offset mean is 168.1299285888672, larger than 100. 19-07-14 21:35:39.045 - WARNING: Offset mean is 8562.78125, larger than 100. 19-07-14 21:35:39.048 - WARNING: Offset mean is 57894.6640625, larger than 100. 19-07-14 21:35:39.053 - WARNING: Offset mean is 167.73036193847656, larger than 100. 19-07-14 21:35:39.054 - WARNING: Offset mean is 8543.248046875, larger than 100. 19-07-14 21:35:39.057 - WARNING: Offset mean is 57762.5, larger than 100. 19-07-14 21:35:39.062 - WARNING: Offset mean is 167.62863159179688, larger than 100. 19-07-14 21:35:39.063 - WARNING: Offset mean is 8538.310546875, larger than 100. 19-07-14 21:35:39.066 - WARNING: Offset mean is 57729.140625, larger than 100. 19-07-14 21:35:39.377 - WARNING: Offset mean is 177.97457885742188, larger than 100. 19-07-14 21:35:39.378 - WARNING: Offset mean is 9058.482421875, larger than 100. 19-07-14 21:35:39.381 - WARNING: Offset mean is 61246.58984375, larger than 100. 19-07-14 21:35:39.386 - WARNING: Offset mean is 178.12855529785156, larger than 100. 19-07-14 21:35:39.387 - WARNING: Offset mean is 9065.642578125, larger than 100. 19-07-14 21:35:39.390 - WARNING: Offset mean is 61294.6796875, larger than 100. 19-07-14 21:35:39.395 - WARNING: Offset mean is 177.98928833007812, larger than 100. 19-07-14 21:35:39.396 - WARNING: Offset mean is 9058.904296875, larger than 100. 19-07-14 21:35:39.399 - WARNING: Offset mean is 61249.17578125, larger than 100. 19-07-14 21:35:39.404 - WARNING: Offset mean is 178.05380249023438, larger than 100. 19-07-14 21:35:39.405 - WARNING: Offset mean is 9062.0673828125, larger than 100. 19-07-14 21:35:39.408 - WARNING: Offset mean is 61270.58203125, larger than 100. 19-07-14 21:35:39.412 - WARNING: Offset mean is 178.0281524658203, larger than 100. 19-07-14 21:35:39.413 - WARNING: Offset mean is 9060.7783203125, larger than 100. 19-07-14 21:35:39.416 - WARNING: Offset mean is 61261.84765625, larger than 100.

Is there anyone have the same problem? It sames like the training is going on.

opened by CaiQiuYu 8
How to properly deal with 'Offset mean more than 100' Warning
Hi Xinntao,

Can you please comment on how to specifically deal with the offset warning in the PCD alignment module? I am trying to train an 'L' model and have ambiguities from the details I could piece together from issues #16 and #22. I have put together the workflow required to train an 'L' model below, can you please comment about its correctness.

The original 'M' and 'L' models have 10 and 40 RBs respectively in the reconstruction module and the number of channels are also different i.e. 64 and 128 respectively.

According to your suggestion of initializing a deeper model with a shallower model, you are referring to using an 'M' model (wo TSA) with 128 feature maps and not with 64? -- as the lateral would not work.

Once the 'M' model (wo TSA) with 128 feature maps is trained, we can initialize the 'L' model (wo TSA) using the 'M' model where for each 10 RBs (out of 40) of 'L' model are initialized with 10 RBs from 'M' model.

Then the 'L' model (wo TSA) can be trained, subsequently we can add TSA module -- train only the TSA module first and then the full network as suggest in the yml file for 'M' model,
opened by TouqeerAhmad 7
Segmentation fault (core dumped) training on 4 gpus

[root@A01-R04-I221-51 codes]# python3 train.py -opt options/train/train_EDVR_woTSA_M_vimeo90k.yml export CUDA_VISIBLE_DEVICES=0,1,2,3 Disabled distributed training. Path already exists. Rename it to [/home/zzt/EDVR/experiments/001_EDVRwoTSA_scratch_lr4e-4_600k_vimeo90k_LrCAR4S_archived_190628-144447] 19-06-28 14:44:47.113 - INFO: name: 001_EDVRwoTSA_scratch_lr4e-4_600k_vimeo90k_LrCAR4S use_tb_logger: True model: VideoSR_base distortion: sr scale: 4 gpu_ids: [0, 1, 2, 3] datasets:[ train:[ name: Vimeo90K mode: Vimeo90K interval_list: [1] random_reverse: False border_mode: False dataroot_GT: /home/zzt/EDVR/datasets/vimeo90k_train_GT.lmdb dataroot_LQ: /home/zzt/EDVR/datasets/vimeo90k_train_LR7frames.lmdb cache_keys: Vimeo90K_train_keys.pkl N_frames: 5 use_shuffle: True n_workers: 3 batch_size: 32 GT_size: 256 LQ_size: 64 use_flip: True use_rot: True color: RGB phase: train scale: 4 data_type: lmdb ] ] network_G:[ which_model_G: EDVR nf: 64 nframes: 5 groups: 8 front_RBs: 5 back_RBs: 10 predeblur: False HR_in: False w_TSA: False scale: 4 ] path:[ pretrain_model_G: None strict_load: True resume_state: None root: /home/zzt/EDVR experiments_root: /home/zzt/EDVR/experiments/001_EDVRwoTSA_scratch_lr4e-4_600k_vimeo90k_LrCAR4S models: /home/zzt/EDVR/experiments/001_EDVRwoTSA_scratch_lr4e-4_600k_vimeo90k_LrCAR4S/models training_state: /home/zzt/EDVR/experiments/001_EDVRwoTSA_scratch_lr4e-4_600k_vimeo90k_LrCAR4S/training_state log: /home/zzt/EDVR/experiments/001_EDVRwoTSA_scratch_lr4e-4_600k_vimeo90k_LrCAR4S val_images: /home/zzt/EDVR/experiments/001_EDVRwoTSA_scratch_lr4e-4_600k_vimeo90k_LrCAR4S/val_images ] train:[ lr_G: 0.0004 lr_scheme: CosineAnnealingLR_Restart beta1: 0.9 beta2: 0.99 niter: 600000 warmup_iter: -1 T_period: [150000, 150000, 150000, 150000] restarts: [150000, 300000, 450000] restart_weights: [1, 1, 1] eta_min: 1e-07 pixel_criterion: cb pixel_weight: 1.0 val_freq: 2000.0 manual_seed: 0 ] logger:[ print_freq: 1 save_checkpoint_freq: 2000.0 ] is_train: True dist: False

19-06-28 14:44:50.625 - INFO: Random seed: 0 19-06-28 14:44:50.628 - INFO: Temporal augmentation interval list: [1], with random reverse is False. 19-06-28 14:44:50.629 - INFO: Using cache keys: Vimeo90K_train_keys.pkl 19-06-28 14:44:50.629 - INFO: Using cache keys - Vimeo90K_train_keys.pkl. 19-06-28 14:44:50.638 - INFO: Dataset [Vimeo90KDataset - Vimeo90K] is created. 19-06-28 14:44:50.638 - INFO: Number of train images: 64,612, iters: 2,020 19-06-28 14:44:50.638 - INFO: Total epochs needed: 298 for iters 600,000 19-06-28 14:44:53.798 - INFO: Network G structure: DataParallel - EDVR, with parameters: 2,996,259 19-06-28 14:44:53.799 - INFO: EDVR( (conv_first): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (feature_extraction): Sequential( (0): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (1): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (2): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (3): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (4): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (fea_L2_conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1)) (fea_L2_conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (fea_L3_conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1)) (fea_L3_conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (pcd_align): PCD_Align( (L3_offset_conv1): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L3_offset_conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L3_dcnpack): DCN_sep( (conv_offset_mask): Conv2d(64, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (L2_offset_conv1): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L2_offset_conv2): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L2_offset_conv3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L2_dcnpack): DCN_sep( (conv_offset_mask): Conv2d(64, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (L2_fea_conv): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L1_offset_conv1): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L1_offset_conv2): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L1_offset_conv3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (L1_dcnpack): DCN_sep( (conv_offset_mask): Conv2d(64, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (L1_fea_conv): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (cas_offset_conv1): Conv2d(128, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (cas_offset_conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (cas_dcnpack): DCN_sep( (conv_offset_mask): Conv2d(64, 216, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (lrelu): LeakyReLU(negative_slope=0.1, inplace) ) (tsa_fusion): Conv2d(320, 64, kernel_size=(1, 1), stride=(1, 1)) (recon_trunk): Sequential( (0): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (1): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (2): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (3): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (4): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (5): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (6): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (7): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (8): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) (9): ResidualBlock_noBN( (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) ) ) (upconv1): Conv2d(64, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (upconv2): Conv2d(64, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (pixel_shuffle): PixelShuffle(upscale_factor=2) (HRconv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (conv_last): Conv2d(64, 3, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (lrelu): LeakyReLU(negative_slope=0.1, inplace) ) 19-06-28 14:44:53.801 - INFO: Model [VideoSRBaseModel] is created. 19-06-28 14:44:53.802 - INFO: Start training from epoch: 0, iter: 0 Segmentation fault (core dumped)

opened by robotzheng 5
Is there a limit to the input image resolution?

this error occurred when I feed 1080p video into the network loaded from "EDVR_REDS_deblurcomp_L.pth". everything is ok on lower resolution video. @xinntao

opened by hahahaprince 5
can anyone meet this error("段错误")

when I run test_Vid4_REDS4_with_GT.py, I got a error . Is anyone can help me to resolve this, thank you very much. I use anaconda3, python 3.6.7 pytorch version is 1.1.0.

python ./test_Vid4_REDS4_with_GT.py 19-05-30 15:01:21.580 - INFO: Data: Vid4 - ../datasets/Vid4/BIx4/* 19-05-30 15:01:21.583 - INFO: Padding mode: new_info 19-05-30 15:01:21.583 - INFO: Model path: ../experiments/pretrained_models/EDVR_Vimeo90K_SR_L.pth 19-05-30 15:01:21.583 - INFO: Save images: True 19-05-30 15:01:21.584 - INFO: Flip Test: False 段错误

I find that the one in line 139("model_output = model(imgs_in)") caused the error, but I did not know why.

opened by huangzhiwei1987 5
can anyone meet this error

when I run, I meet this error. May be pytoch version . Is anyone can help me to resolve this, thank you very much. I use anaconda3, pytorch version is 1.0.1.

opened by yanmenglu 5
AttributeError: module 'tensorflow' has no attribute 'io'

When I run the python train.py -opt options/train/train_SRResNet.yml I got the following error

export CUDA_VISIBLE_DEVICES=0 Disabled distributed training. 19-11-11 17:11:42.438 - INFO: name: 001_MSRResNetx4_scratch_DIV2K use_tb_logger: True model: sr distortion: sr scale: 4 gpu_ids: [0] datasets:[ train:[ name: DIV2K mode: LQGT dataroot_GT: ../datasets/DIV2K/DIV2K800_sub.lmdb dataroot_LQ: ../datasets/DIV2K/DIV2K800_sub_bicLRx4.lmdb use_shuffle: True n_workers: 6 batch_size: 16 GT_size: 128 use_flip: True use_rot: True color: RGB phase: train scale: 4 data_type: lmdb ] val:[ name: val_set5 mode: LQGT dataroot_GT: ../datasets/val_set5/Set5 dataroot_LQ: ../datasets/val_set5/Set5_bicLRx4 phase: val scale: 4 data_type: img ] ] network_G:[ which_model_G: MSRResNet in_nc: 3 out_nc: 3 nf: 64 nb: 16 upscale: 4 scale: 4 ] path:[ pretrain_model_G: None strict_load: True resume_state: None root: /home/lijunnian/Desktop/image_Debluring/EDVR experiments_root: /home/lijunnian/Desktop/image_Debluring/EDVR/experiments/001_MSRResNetx4_scratch_DIV2K models: /home/lijunnian/Desktop/image_Debluring/EDVR/experiments/001_MSRResNetx4_scratch_DIV2K/models training_state: /home/lijunnian/Desktop/image_Debluring/EDVR/experiments/001_MSRResNetx4_scratch_DIV2K/training_state log: /home/lijunnian/Desktop/image_Debluring/EDVR/experiments/001_MSRResNetx4_scratch_DIV2K val_images: /home/lijunnian/Desktop/image_Debluring/EDVR/experiments/001_MSRResNetx4_scratch_DIV2K/val_images ] train:[ lr_G: 0.0002 lr_scheme: CosineAnnealingLR_Restart beta1: 0.9 beta2: 0.99 niter: 1000000 warmup_iter: -1 T_period: [250000, 250000, 250000, 250000] restarts: [250000, 500000, 750000] restart_weights: [1, 1, 1] eta_min: 1e-07 pixel_criterion: l1 pixel_weight: 1.0 manual_seed: 10 val_freq: 5000.0 ] logger:[ print_freq: 100 save_checkpoint_freq: 5000.0 ] is_train: True dist: False

/usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, **kwds) /usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, **kwds) /usr/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 return f(*args, **kwds) Traceback (most recent call last): File "train.py", line 310, in main() File "train.py", line 81, in main tb_logger = SummaryWriter(log_dir='../tb_logger/' + opt['name']) File "/usr/local/lib/python3.5/dist-packages/torch/utils/tensorboard/writer.py", line 199, in init self._get_file_writer() File "/usr/local/lib/python3.5/dist-packages/torch/utils/tensorboard/writer.py", line 248, in _get_file_writer self.file_writer = FileWriter(logdir=self.log_dir, **self.kwargs) File "/usr/local/lib/python3.5/dist-packages/torch/utils/tensorboard/writer.py", line 65, in init logdir, max_queue, flush_secs, filename_suffix) File "/usr/local/lib/python3.5/dist-packages/tensorboard/summary/writer/event_file_writer.py", line 73, in init if not tf.io.gfile.exists(logdir): File "/usr/local/lib/python3.5/dist-packages/tensorboard/lazy.py", line 65, in getattr return getattr(load_once(self), attr_name) AttributeError: module 'tensorflow' has no attribute 'io'

python 3.5.2 Pytorch 1.0.1 GCC 5.4.0 CUDA 9.0 what should I do to handle the error? thanks in advance.

opened by BrainPotter 4
what is the mean of the mask

hi, what is the purpose of generating the mask?

https://github.com/xinntao/EDVR/blob/b02e63a0fc5854cad7b11a87f74601612e356eff/basicsr/models/archs/arch_util.py#L245

opened by Jee-King 0
EDVR colab?

Will there be an addition of a EDVR video de-blurring Colab any time soon? I'm having a difficult time trying to figure out how I would test EDVR and a Colab notebook would come in handy for this. Thanks!

opened by noobtoob4lyfe 5
Ringing artifacts on the output

Hello.. I was training EDVR with pre-deblur and TSA modules. I used the REDS4 dataset with some degradations from Real-ESRGAN such as random downsampling, blurring and different noises such as Gaussian, Poisson, JPEG etc. I am now getting some ringing artifacts on the testing data (which is blurry and noisy as well). The ringing artifacts are more in the middle region, but comparatively less in the top region as shown below, I think that the reason for these artifacts is the pre-deblur block. What maybe the issue I am facing here? What could be the solution for this problem?

Thank you in advance.

opened by shivarama93 0
EDVR: num_out_ch completely useless

Hello,

In the code edvr_arch.py for the architecture EDVR there is an argument call num_out_ch for Channel number of output image, but this argument is not used and so its always 3 channel out even if we put num_out_ch=1.

Thanks.

opened by Scienceseb 0

Owner

Xintao

Researcher at Tencent ARC Lab, (Applied Research Center)

GitHub https://github.com/xinntao/BasicSR

3D2Unet: 3D Deformable Unet for Low-Light Video Enhancement (PRCV2021)

3DDUNET This is the code for 3D2Unet: 3D Deformable Unet for Low-Light Video Enhancement (PRCV2021) Conference Paper Link Dataset We use SMOID dataset

1 Jan 7, 2022

CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhancement

CBREN This is the Pytorch implementation for our IEEE TCSVT paper : CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhanceme

3 Nov 4, 2022

Winning solution of the Indoor Location & Navigation Kaggle competition

This repository contains the code to generate the winning solution of the Kaggle competition on indoor location and navigation organized by Microsoft

62 Dec 28, 2022

This is the winning solution of the Endocv-2021 grand challange.

Endocv2021-winner [Paper] This is the winning solution of the Endocv-2021 grand challange. Dependencies pytorch # tested with 1.7 and 1.8 torchvision

14 Dec 3, 2022

The sixth place winning solution (6/220) in 2021 Gaofen Challenge.

SwinTransformer + OBBDet The sixth place winning solution (6/220) in the track of Fine-grained Object Recognition in High-Resolution Optical Images, 2

46 Dec 2, 2022

A Research-oriented Federated Learning Library and Benchmark Platform for Graph Neural Networks. Accepted to ICLR'2021 - DPML and MLSys'21 - GNNSys workshops.

FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks A Research-oriented Federated Learning Library and Benchmark Platform

175 Dec 1, 2022

EDPN: Enhanced Deep Pyramid Network for Blurry Image Restoration

EDPN: Enhanced Deep Pyramid Network for Blurry Image Restoration Ruikang Xu, Zeyu Xiao, Jie Huang, Yueyi Zhang, Zhiwei Xiong. EDPN: Enhanced Deep Pyra

69 Dec 15, 2022

The 7th edition of NTIRE: New Trends in Image Restoration and Enhancement workshop will be held on June 2022 in conjunction with CVPR 2022.

NTIRE 2022 - Image Inpainting Challenge Important dates 2022.02.01: Release of train data (input and output images) and validation data (only input) 2

37 Nov 27, 2022

A custom DeepStack model that has been trained detecting ONLY the USPS logo

This repository provides a custom DeepStack model that has been trained detecting ONLY the USPS logo. This was created after I discovered that the Deepstack OpenLogo custom model I was using did not contain USPS.

9 Dec 27, 2022

Some code of the implements of Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network

3D-GMPDCNN Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network PyTorch implementation of "Geological Modeling Usin

5 Nov 21, 2022

This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of Coordinate Independent Convolutional Networks.

Orientation independent Möbius CNNs This repository implements and evaluates convolutional networks on the Möbius strip as toy model instantiations of

59 Dec 9, 2022