Official pytorch implementation of the IrwGAN for unaligned image-to-image translation

Overview

IrwGAN (ICCV2021)

Unaligned Image-to-Image Translation by Learning to Reweight

[Update] 12/15/2021 All dataset are released, trained models and generated images of IrwGAN are released

[Update] 11/16/2021 Code is pushed, selfie2anime-danbooru dataset released.

Dataset

selfie2anime-danbooru | selfie-horse2zebra-dog | horse-cat2dog-anime | beetle-tiger2lion-sealion

Trained Models and Generated Images

  • selfie2anime-danbooru   IrwGAN | [Baseline] | [CycleGAN] | [MUNIT] | [GcGAN] | [NICE-GAN]
  • selfie-horse2zebra-dog   IrwGAN | [Baseline] | [CycleGAN] | [MUNIT] | [GcGAN] | [NICE-GAN]
  • horse-cat2dog-anime     IrwGAN | [Baseline] | [CycleGAN] | [MUNIT] | [GcGAN] | [NICE-GAN]
  • beetle-tiger2lion-sealion IrwGAN | [Baseline] | [CycleGAN] | [MUNIT] | [GcGAN] | [NICE-GAN]

Basic Usage

  • Training:
python main.py --dataroot=datasets/selfie2anime-danbooru 
  • Resume:
python main.py --dataroot=datasets/selfie2anime-danbooru --phase=resume
  • Test:
python main.py --dataroot=datasets/selfie2anime-danbooru --phase=test
  • Beta Mode --beta_mode=A if domain A is unaligned, --beta_mode=B if domain B is unaligned, --beta_mode=AB if two domains are unaligned
  • Effective Sample Size lambda_nos_A and lambda_nos_B are used to control how many samples are selected. The higher the weight, more samples are selected. We use 1.0 across all experiments.

Example Results

Citation

If you use this code for your research, please cite our paper:

@inproceedings{xie2021unaligned,
  title={Unaligned Image-to-Image Translation by Learning to Reweight},
  author={Xie, Shaoan and Gong, Mingming and Xu, Yanwu and Zhang, Kun},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={14174--14184},
  year={2021}
}
Issues
  • testing error

    testing error

    Hello, I am a deep learning newbie currently testing your model on a dataset as a proposal for an MS project but when I run it using the provided code, I get this: /usr/local/lib/python3.7/dist-packages/torchvision/transforms/transforms.py:333: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum. "Argument interpolation should be of type InterpolationMode instead of int. " model [IRWGANModel] was created

    Information

    task: dibcoirwgan

    phase: test

    gan_type: lsgan

    netD: gl

    trainA_size: 93

    trainB_size: 84

    testA_size: 16

    testB_size: 16

    Weight

    lambda_A: 10

    lambda_B: 10

    lambda_identity: 1

    Model Specific

    beta_mode: AB

    threshold: 0.1

    batch_size: 20

    lambda_nos_A: 1

    lambda_nos_B: 1

    -------------- Networks loaded ---------------- [Network gen_a2b] Total number of parameters : 11.379 M [Network gen_b2a] Total number of parameters : 11.379 M [Network dis_a] Total number of parameters : 13.969 M [Network dis_b] Total number of parameters : 13.969 M [Network beta_net_a] Total number of parameters : 2.757 M [Network beta_net_b] Total number of parameters : 2.757 M

    [*] testing start! Traceback (most recent call last): File "main.py", line 117, in test(model, opt, test_loader_a, test_loader_b) File "main.py", line 38, in test dict_a, dict_b = misc.test_fid(test_loader_a, model.gen_a2b, test_loader_b, model.gen_b2a, model.run_dir, opt) File "/content/drive/MyDrive/Colab Notebooks/IrwGAN/models/misc.py", line 125, in test_fid metric_dict_AB = torch_fidelity.calculate_metrics(input1=real_b_path, input2=fake_b_path, **eval_args) File "/usr/local/lib/python3.7/dist-packages/torch_fidelity/metrics.py", line 239, in calculate_metrics featuresdict_1 = extract_featuresdict_from_input_id_cached(1, feat_extractor, **kwargs) File "/usr/local/lib/python3.7/dist-packages/torch_fidelity/utils.py", line 372, in extract_featuresdict_from_input_id_cached featuresdict = fn_recompute() File "/usr/local/lib/python3.7/dist-packages/torch_fidelity/utils.py", line 360, in fn_recompute return extract_featuresdict_from_input_id(input_id, feat_extractor, **kwargs) File "/usr/local/lib/python3.7/dist-packages/torch_fidelity/utils.py", line 342, in extract_featuresdict_from_input_id input = prepare_input_from_id(input_id, **kwargs) File "/usr/local/lib/python3.7/dist-packages/torch_fidelity/utils.py", line 275, in prepare_input_from_id return prepare_input_from_descriptor(input_desc, **kwargs) File "/usr/local/lib/python3.7/dist-packages/torch_fidelity/utils.py", line 230, in prepare_input_from_descriptor vassert(len(input) > 0, f'No samples found in {input} with samples_find_deep={samples_find_deep}') File "/usr/local/lib/python3.7/dist-packages/torch_fidelity/helpers.py", line 9, in vassert raise ValueError(message) ValueError: No samples found in [] with samples_find_deep=False

    has anyone ever come across this?

    opened by jrdyhwh 1
  • Colab implementation

    Colab implementation

    Hi, Thank you for the great work.

    Due to the limited computation power, I can't run your code directly. I'm wondering if it is possible that you could have colab implementation of your work?

    Thank you:)

    opened by akamob 2
Maximum Spatial Perturbation for Image-to-Image Translation (Official Implementation)

MSPC for I2I This repository is by Yanwu Xu and contains the PyTorch source code to reproduce the experiments in our CVPR2022 paper Maximum Spatial Pe

null 41 Aug 5, 2022
This is the PyTorch implementation of GANs N’ Roses: Stable, Controllable, Diverse Image to Image Translation

Official PyTorch repo for GAN's N' Roses. Diverse im2im and vid2vid selfie to anime translation.

null 1k Aug 11, 2022
PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Networks".

pix2pix-pytorch PyTorch implementation of Image-to-Image Translation Using Conditional Adversarial Networks. Based on pix2pix by Phillip Isola et al.

mrzhu 375 Aug 10, 2022
Image-to-Image Translation in PyTorch

CycleGAN and pix2pix in PyTorch New: Please check out contrastive-unpaired-translation (CUT), our new unpaired image-to-image translation model that e

Jun-Yan Zhu 18.1k Aug 12, 2022
Official implementation for "QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation" (CVPR 2022)

QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation (CVPR2022) https://arxiv.org/abs/2203.08483 Unpaired image-to-image (I2I

Xueqi Hu 34 Jul 27, 2022
Image-to-Image Translation with Conditional Adversarial Networks (Pix2pix) implementation in keras

pix2pix-keras Pix2pix implementation in keras. Original paper: Image-to-Image Translation with Conditional Adversarial Networks (pix2pix) Paper Author

William Falcon 138 Jul 4, 2022
This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

RGB2NIR_Experimental This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models

null 2 Dec 14, 2021
Pytorch implementation of our method for high-resolution (e.g. 2048x1024) photorealistic video-to-video translation.

vid2vid Project | YouTube(short) | YouTube(full) | arXiv | Paper(full) Pytorch implementation for high-resolution (e.g., 2048x1024) photorealistic vid

NVIDIA Corporation 8k Aug 6, 2022
PyTorch Implementation of "Non-Autoregressive Neural Machine Translation"

Non-Autoregressive Transformer Code release for Non-Autoregressive Neural Machine Translation by Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K.

Salesforce 254 Aug 16, 2022
Official code of our work, AVATAR: A Parallel Corpus for Java-Python Program Translation.

AVATAR Official code of our work, AVATAR: A Parallel Corpus for Java-Python Program Translation. AVATAR stands for jAVA-pyThon progrAm tRanslation. AV

Wasi Ahmad 23 Jun 30, 2022
Code for Dual Contrastive Learning for Unsupervised Image-to-Image Translation, NTIRE, CVPRW 2021.

arXiv Dual Contrastive Learning Adversarial Generative Networks (DCLGAN) We provide our PyTorch implementation of DCLGAN, which is a simple yet powerf

null 106 Jul 29, 2022
Unsupervised Image-to-Image Translation

UNIT: UNsupervised Image-to-image Translation Networks Imaginaire Repository We have a reimplementation of the UNIT method that is more performant. It

Ming-Yu Liu 劉洺堉 1.9k Aug 10, 2022
Breaking the Dilemma of Medical Image-to-image Translation

Breaking the Dilemma of Medical Image-to-image Translation Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field

Kid Liet 72 Aug 8, 2022
Implementation of "Glancing Transformer for Non-Autoregressive Neural Machine Translation"

GLAT Implementation for the ACL2021 paper "Glancing Transformer for Non-Autoregressive Neural Machine Translation" Requirements Python >= 3.7 Pytorch

null 109 Aug 8, 2022
Official PyTorch implementation of "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows"

ArtFlow Official PyTorch implementation of the paper: ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows Jie An*, Siyu Huang*, Yibing

null 119 Jul 12, 2022
Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN", accepted to ACM MM 2021 BNI Track.

RecycleD Official PyTorch implementation of the paper "Recycling Discriminator: Towards Opinion-Unaware Image Quality Assessment Using Wasserstein GAN

Yunan Zhu 21 Jul 14, 2022
[CVPR 2021] Official PyTorch Implementation for "Iterative Filter Adaptive Network for Single Image Defocus Deblurring"

IFAN: Iterative Filter Adaptive Network for Single Image Defocus Deblurring Checkout for the demo (GUI/Google Colab)! The GUI version might occasional

Junyong Lee 144 Aug 4, 2022
Official PyTorch implementation of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image", ICCV 2019

PoseNet of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image" Introduction This repo is official Py

Gyeongsik Moon 634 Aug 7, 2022
[BMVC 2021] Official PyTorch Implementation of Self-supervised learning of Image Scale and Orientation Estimation

Self-Supervised Learning of Image Scale and Orientation Estimation (BMVC 2021) This is the official implementation of the paper "Self-Supervised Learn

Jongmin Lee 13 Jul 20, 2022