TSIT: A Simple and Versatile Framework for Image-to-Image Translation

Overview

TSIT: A Simple and Versatile Framework for Image-to-Image Translation

teaser

This repository provides the official PyTorch implementation for the following paper:

TSIT: A Simple and Versatile Framework for Image-to-Image Translation
Liming Jiang, Changxu Zhang, Mingyang Huang, Chunxiao Liu, Jianping Shi and Chen Change Loy
In ECCV 2020 (Spotlight).
Paper

Abstract: We introduce a simple and versatile framework for image-to-image translation. We unearth the importance of normalization layers, and provide a carefully designed two-stream generative model with newly proposed feature transformations in a coarse-to-fine fashion. This allows multi-scale semantic structure information and style representation to be effectively captured and fused by the network, permitting our method to scale to various tasks in both unsupervised and supervised settings. No additional constraints (e.g., cycle consistency) are needed, contributing to a very clean and simple method. Multi-modal image synthesis with arbitrary style control is made possible. A systematic study compares the proposed method with several state-of-the-art task-specific baselines, verifying its effectiveness in both perceptual quality and quantitative evaluations.

Updates

  • [01/2021] The code of TSIT is released.

  • [07/2020] The paper of TSIT is accepted by ECCV 2020 (Spotlight).

Installation

After installing Anaconda, we recommend you to create a new conda environment with python 3.7.6:

conda create -n tsit python=3.7.6 -y
conda activate tsit

Clone this repo, install PyTorch 1.1.0 (newer versions may also work) and other dependencies:

git clone https://github.com/EndlessSora/TSIT.git
cd TSIT
pip install -r requirements.txt

This code also requires the Synchronized-BatchNorm-PyTorch:

cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
rm -rf Synchronized-BatchNorm-PyTorch
cd ../../

Tasks and Datasets

The code covers 3 image-to-image translation tasks on 5 datasets. For more details, please refer to our paper.

Task Abbreviations

  • Arbitrary Style Transfer (AST) on Yosemite summer → winter, BDD100K day → night, and Photo → art datasets.
  • Semantic Image Synthesis (SIS) on Cityscapes and ADE20K datasets.
  • Multi-Modal Image Synthesis (MMIS) on BDD100K sunny → different time/weather conditions dataset.

The abbreviations are used to specify the --task argument when training and testing.

Dataset Preparation

We provide one-click scripts to prepare datasets. The details are provided below.

  • Yosemite summer → winter and Photo → art. The provided scripts will make all things ready (including the download). For example, simply run:
bash datasets/prepare_summer2winteryosemite.sh
  • BDD100K. Please first download BDD100K Images on their official website. We have provided the classified lists of different weathers and times. After downloading, you only need to run:
bash datasets/prepare_bdd100k.sh [data_root]

The [data_root] should be specified, which is the path to the BDD100K root folder that contains images folder. The script will put the list to the suitable place and symlink the root folder to ./datasets.

  • Cityscapes. Please follow the standard download and preparation guidelines on the official website. We recommend to symlink its root folder [data_root] to ./datasets by:
bash datasets/prepare_cityscapes.sh [data_root]
  • ADE20K. The dataset can be downloaded here, which is from MIT Scene Parsing BenchMark. After unzipping the dataset, put the jpg image files ADEChallengeData2016/images/ and png label files ADEChallengeData2016/annotatoins/ in the same directory. We also recommend to symlink its root folder [data_root] to ./datasets by:
bash datasets/prepare_ade20k.sh [data_root]

Testing Pretrained Models

  1. Download the pretrained models and unzip them to ./checkpoints.

  2. For a quick start, we have provided all the example test scripts. After preparing the corresponding datasets, you can directly use the test scripts. For example:

bash test_scripts/ast_summer2winteryosemite.sh
  1. The generated images will be saved at ./results/[experiment_name] by default.

  2. You can use --results_dir to specify the output directory. --how_many will specify the maximum number of images to generate. By default, the code loads the latest checkpoint, which can be changed using --which_epoch. You can also discard --show_input to show the generated images only without the input references.

  3. For MMIS sunny → different time/weather conditions, the --test_mode can be specified (optional): night | cloudy | rainy | snowy | all (default).

Training

For a quick start, we have provided all the example training scripts. After preparing the corresponding datasets, you can directly use the training scripts. For example:

bash train_scripts/ast_summer2winteryosemite.sh

Please note that you may want to change the experiment name --name or the checkpoint saving root --checkpoints_dir to prevent your newly trained models overwriting the pretrained ones (if used).

--task is given using the abbreviations. --dataset_mode specifies the dataset type. --croot and --sroot specify the content and style data root, respectively. The results may be better reproduced on NVIDIA Tesla V100 GPUs.

After training, testing the newly trained models is similar to testing pretrained models.

Citation

If you find this work useful for your research, please cite our paper:

@inproceedings{jiang2020tsit,
  title={{TSIT}: A Simple and Versatile Framework for Image-to-Image Translation},
  author={Jiang, Liming and Zhang, Changxu and Huang, Mingyang and Liu, Chunxiao and Shi, Jianping and Loy, Chen Change},
  booktitle={ECCV},
  year={2020}
}

Acknowledgments

The code is greatly inspired by SPADE, pytorch-AdaIN, and Synchronized-BatchNorm-PyTorch.

License

Copyright (c) 2020. All rights reserved.

The code is licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International).

Comments
  • three question about network detail

    three question about network detail

    Hello, thanks your great paper first. I am trying to reproduce your work. I have three question while coding.

    1. What is the network structure of input layer and output layer, who we put an input layer at the entrance. The feature channel is adjusted to 64 after the input layer, while the resolution remains unchanged. and output layer who reduce 64 channels feature to image?

    for now, I use conv block:

    # input layer
    nn.Sequential(
        conv_block(self.use_spectral, in_channels, out_channels, kernel_size=7, stride=1,
                           padding_mode="zeros", padding=3, bias=True),
        nn.InstanceNorm2d(out_channels),
        nn.ReLU(inplace=True)
    )
    # output layer
    nn.Sequential(
        conv_block(use_spectral, base_channels, out_channels, kernel_size=7, padding=3, padding_mode="zeros"),
        nn.Tanh()
    )
    
    1. In your network, FAdaIN is placed at the front of the BatchNorm(in FADE). While do arbitrary style transfer task, the batch size is 1, so that BatchNorm have the same behavior as instance norm(Is this correct?), Didn’t the sytle injected by FAdaIN disappear?

    related code in my implementation:

    # main code in AdaIN
    self.norm = nn.InstanceNorm2d(num_features, eps, momentum, affine, track_running_stats)
    x = self.gamma * x + self.beta
    
    # code in FADE,
    nn.BatchNorm2d(num_features=in_channels, affine=False, track_running_stats=True)
    
    1. Despite the above two questions, I have tried my implementation in a unpaired dataset( 6000 human face to 3400 anime face). and I got this result (during training after ~90000 pairs): image

    content image style image generated image

    batch size = 1, feature matching loss weight = 1, perceptual loss weight = 1

    All images generated in test dataset. It seems that the generated image have the average style, but not the style of the input style image. Is this a problem with my implementation, or a problem with the network in paper?

    opened by budui 5
  • cuda runtime error (700) : an illegal memory access was encountered

    cuda runtime error (700) : an illegal memory access was encountered

    I try to test your code with a custom dataset using the following command:

    python train.py \
        --name sis_maps \
        --task SIS \
        --gpu_ids 0,1 \
        --checkpoints_dir  ./checkpoints\
        --batchSize 4 \
        --dataset_mode custom \
        --croot ../../datasets/maps \
        --sroot ../../datasets/maps \
        --nThreads 4 \
        --gan_mode hinge \
        --num_upsampling_layers more \
        --use_vae \
        --alpha 1.0 \
        --display_freq 1000 \
        --save_epoch_freq 20 \
        --niter 100 \
        --niter_decay 100 \
        --lambda_vgg 20 \
        --lambda_feat 10 \
        --no_instance \
        --label_dir ../../datasets/maps/trainA \
        --image_dir ../../datasets/maps/trainB
    

    The goal is to translate images from the domain of trainA into images from the domain of trainB. Do I understand the flags label_dir and image_dir correctly?

    The dataset and network are created without throwing an error:

    dataset [CustomDataset] of size 1096 was created
    Network [TSITGenerator] was created. Total number of parameters: 332.4 million. To see the architecture, do print(network).
    Network [MultiscaleDiscriminator] was created. Total number of parameters: 5.6 million. To see the architecture, do print(network).
    Network [ConvEncoder] was created. Total number of parameters: 10.5 million. To see the architecture, do print(network).
    create web directory ./checkpoints/sis_maps/web...
      0%|                                                                                  | 0/200 [00:00<?, ?it/sTHCudaCheck FAIL file=/pytorch/aten/src/THC/generic/THCTensorScatterGather.cu line=384 error=700 : an illegal memory access was encountered
    

    However, the code fails when it tries to create a web directory:

    Traceback (most recent call last):
      File "train.py", line 37, in <module>
        trainer.run_generator_one_step(data_i)
      File "/net/coxfs01/srv/export/coxfs01/pfister_lab2/share_root/Lab/leander/tsit/TSIT/trainers/pix2pix_trainer.py", line 30, in run_generator_one_step
        g_losses, generated = self.pix2pix_model(data, mode='generator')
      File "/n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
        result = self.forward(*input, **kwargs)
      File "/n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 153, in forward
        return self.module(*inputs[0], **kwargs[0])
      File "/n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
        result = self.forward(*input, **kwargs)
      File "/net/coxfs01/srv/export/coxfs01/pfister_lab2/share_root/Lab/leander/tsit/TSIT/models/pix2pix_model.py", line 37, in forward
        input_semantics, real_image = self.preprocess_input(data)
      File "/net/coxfs01/srv/export/coxfs01/pfister_lab2/share_root/Lab/leander/tsit/TSIT/models/pix2pix_model.py", line 118, in preprocess_input
        input_semantics = input_label.scatter_(1, label_map, 1.0)
    RuntimeError: cuda runtime error (700) : an illegal memory access was encountered at /pytorch/aten/src/THC/generic/THCTensorScatterGather.cu:384
    terminate called after throwing an instance of 'c10::Error'
      what():  CUDA error: an illegal memory access was encountered (insert_events at /pytorch/c10/cuda/CUDACachingAllocator.cpp:771)
    frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x46 (0x2b41150c3536 in /n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/lib/libc10.so)
    frame #1: c10::cuda::CUDACachingAllocator::raw_delete(void*) + 0x7ae (0x2b4114e7dfbe in /n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/lib/libc10_cuda.so)
    frame #2: c10::TensorImpl::release_resources() + 0x4d (0x2b41150b3abd in /n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/lib/libc10.so)
    frame #3: <unknown function> + 0x5236b2 (0x2b40c90696b2 in /n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
    frame #4: <unknown function> + 0x523756 (0x2b40c9069756 in /n/home05/lauenburg/.conda/envs/tsit/lib/python3.7/site-packages/torch/lib/libtorch_python.so)
    <omitting python frames>
    frame #20: __libc_start_main + 0xf5 (0x2b40c08cd555 in /lib64/libc.so.6)
    

    I tried resolving the issue by following e.g.:

    • https://github.com/pytorch/pytorch/issues/21819
    • https://github.com/pytorchbearer/torchbearer/issues/658#issuecomment-581407150 But I could not resolve it.
    opened by Lauenburg 3
  • cityscapes sis pretrained model

    cityscapes sis pretrained model

    I am getting an error when I run test script for cityscapes sis

    The loaded model is missing keys

    Missing key(s) in state_dict: "style_stream.res_0.conv_0.weight_orig", "style_stream.res_0.conv_0.weight", "style_stream.res_0.conv_0.weight_u", "style_stream.res_0.conv_0.bias", "style_stream.res_0.conv_0.weight_orig", "style_stream.res_0.conv_0.weight_u", "style_stream.res_0.conv_0.weight_v", "style_stream.res_0.conv_1.weight_orig", "style_stream.res_0.conv_1.weight", "style_stream.res_0.conv_1.weight_u", "style_stream.res_0.conv_1.bias", "style_stream.res_0.conv_1.weight_orig", "style_stream.res_0.conv_1.weight_u", "style_stream.res_0.conv_1.weight_v", "style_stream.res_0.conv_s.weight_orig", "style_stream.res_0.conv_s.weight", "style_stream.res_0.conv_s.weight_u", "style_stream.res_0.conv_s.weight_orig", "style_stream.res_0.conv_s.weight_u", "style_stream.res_0.conv_s.weight_v", "style_stream.res_1.conv_0.weight_orig", "style_stream.res_1.conv_0.weight", "style_stream.res_1.conv_0.weight_u", "style_stream.res_1.conv_0.bias", "style_stream.res_1.conv_0.weight_orig", "style_stream.res_1.conv_0.weight_u", "style_stream.res_1.conv_0.weight_v", "style_stream.res_1.conv_1.weight_orig", "style_stream.res_1.conv_1.weight", "style_stream.res_1.conv_1.weight_u", "style_stream.res_1.conv_1.bias", "style_stream.res_1.conv_1.weight_orig", "style_stream.res_1.conv_1.weight_u", "style_stream.res_1.conv_1.weight_v", "style_stream.res_1.conv_s.weight_orig", "style_stream.res_1.conv_s.weight", "style_stream.res_1.conv_s.weight_u", "style_stream.res_1.conv_s.weight_orig", "style_stream.res_1.conv_s.weight_u", "style_stream.res_1.conv_s.weight_v", "style_stream.res_2.conv_0.weight_orig", "style_stream.res_2.conv_0.weight", "style_stream.res_2.conv_0.weight_u", "style_stream.res_2.conv_0.bias", "style_stream.res_2.conv_0.weight_orig", "style_stream.res_2.conv_0.weight_u", "style_stream.res_2.conv_0.weight_v", "style_stream.res_2.conv_1.weight_orig", "style_stream.res_2.conv_1.weight", "style_stream.res_2.conv_1.weight_u", "style_stream.res_2.conv_1.bias", "style_stream.res_2.conv_1.weight_orig", "style_stream.res_2.conv_1.weight_u", "style_stream.res_2.conv_1.weight_v", "style_stream.res_2.conv_s.weight_orig", "style_stream.res_2.conv_s.weight", "style_stream.res_2.conv_s.weight_u", "style_stream.res_2.conv_s.weight_orig", "style_stream.res_2.conv_s.weight_u", "style_stream.res_2.conv_s.weight_v", "style_stream.res_3.conv_0.weight_orig", "style_stream.res_3.conv_0.weight", "style_stream.res_3.conv_0.weight_u", "style_stream.res_3.conv_0.bias", "style_stream.res_3.conv_0.weight_orig", "style_stream.res_3.conv_0.weight_u", "style_stream.res_3.conv_0.weight_v", "style_stream.res_3.conv_1.weight_orig", "style_stream.res_3.conv_1.weight", "style_stream.res_3.conv_1.weight_u", "style_stream.res_3.conv_1.bias", "style_stream.res_3.conv_1.weight_orig", "style_stream.res_3.conv_1.weight_u", "style_stream.res_3.conv_1.weight_v", "style_stream.res_3.conv_s.weight_orig", "style_stream.res_3.conv_s.weight", "style_stream.res_3.conv_s.weight_u", "style_stream.res_3.conv_s.weight_orig", "style_stream.res_3.conv_s.weight_u", "style_stream.res_3.conv_s.weight_v", "style_stream.res_4.conv_0.weight_orig", "style_stream.res_4.conv_0.weight", "style_stream.res_4.conv_0.weight_u", "style_stream.res_4.conv_0.bias", "style_stream.res_4.conv_0.weight_orig", "style_stream.res_4.conv_0.weight_u", "style_stream.res_4.conv_0.weight_v", "style_stream.res_4.conv_1.weight_orig", "style_stream.res_4.conv_1.weight", "style_stream.res_4.conv_1.weight_u", "style_stream.res_4.conv_1.bias", "style_stream.res_4.conv_1.weight_orig", "style_stream.res_4.conv_1.weight_u", "style_stream.res_4.conv_1.weight_v", "style_stream.res_4.conv_s.weight_orig", "style_stream.res_4.conv_s.weight", "style_stream.res_4.conv_s.weight_u", "style_stream.res_4.conv_s.weight_orig", "style_stream.res_4.conv_s.weight_u", "style_stream.res_4.conv_s.weight_v", "style_stream.res_5.conv_0.weight_orig", "style_stream.res_5.conv_0.weight", "style_stream.res_5.conv_0.weight_u", "style_stream.res_5.conv_0.bias", "style_stream.res_5.conv_0.weight_orig", "style_stream.res_5.conv_0.weight_u", "style_stream.res_5.conv_0.weight_v", "style_stream.res_5.conv_1.weight_orig", "style_stream.res_5.conv_1.weight", "style_stream.res_5.conv_1.weight_u", "style_stream.res_5.conv_1.bias", "style_stream.res_5.conv_1.weight_orig", "style_stream.res_5.conv_1.weight_u", "style_stream.res_5.conv_1.weight_v", "style_stream.res_6.conv_0.weight_orig", "style_stream.res_6.conv_0.weight", "style_stream.res_6.conv_0.weight_u", "style_stream.res_6.conv_0.bias", "style_stream.res_6.conv_0.weight_orig", "style_stream.res_6.conv_0.weight_u", "style_stream.res_6.conv_0.weight_v", "style_stream.res_6.conv_1.weight_orig", "style_stream.res_6.conv_1.weight", "style_stream.res_6.conv_1.weight_u", "style_stream.res_6.conv_1.bias", "style_stream.res_6.conv_1.weight_orig", "style_stream.res_6.conv_1.weight_u", "style_stream.res_6.conv_1.weight_v", "style_stream.res_7.conv_0.weight_orig", "style_stream.res_7.conv_0.weight", "style_stream.res_7.conv_0.weight_u", "style_stream.res_7.conv_0.bias", "style_stream.res_7.conv_0.weight_orig", "style_stream.res_7.conv_0.weight_u", "style_stream.res_7.conv_0.weight_v", "style_stream.res_7.conv_1.weight_orig", "style_stream.res_7.conv_1.weight", "style_stream.res_7.conv_1.weight_u", "style_stream.res_7.conv_1.bias", "style_stream.res_7.conv_1.weight_orig", "style_stream.res_7.conv_1.weight_u", "style_stream.res_7.conv_1.weight_v".

    opened by aarjunsrinivasan 3
  • The result may not correct.

    The result may not correct.

    Thanks for sharing this amazing paper. However, I think the accu of Cityscapes may not correct (94.4 in Table 2 of your paper) according to https://github.com/xh-liu/CC-FPSE/issues/4. Can you double-check it? Thanks.

    opened by Ha0Tang 3
  • latest_net_G.pth

    latest_net_G.pth

    Thank you very much for your excellent work.I want to use your Sunny2Diffweather model to make Finetune, which requires the parameters of G and D. However, you only provided the parameter of G, so I would like to request it. Just need SUNNY2DIFFWEATHER LATEST_NET_D.PTH, thank you very much!!

    opened by xiaoachen98 2
  • Bump pillow from 8.2.0 to 8.3.2

    Bump pillow from 8.2.0 to 8.3.2

    ⚠️ Dependabot is rebasing this PR ⚠️

    Rebasing might not happen immediately, so don't worry if this takes some time.

    Note: if you make any changes to this PR yourself, they will take precedence over the rebase.


    Bumps pillow from 8.2.0 to 8.3.2.

    Release notes

    Sourced from pillow's releases.

    8.3.2

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.2.html

    Security

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    Python 3.10 wheels

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    Fixed regressions

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.1.html

    Changes

    8.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.3.2 (2021-09-02)

    • CVE-2021-23437 Raise ValueError if color specifier is too long [hugovk, radarhere]

    • Fix 6-byte OOB read in FliDecode [wiredfool]

    • Add support for Python 3.10 #5569, #5570 [hugovk, radarhere]

    • Ensure TIFF RowsPerStrip is multiple of 8 for JPEG compression #5588 [kmilos, radarhere]

    • Updates for ImagePalette channel order #5599 [radarhere]

    • Hide FriBiDi shim symbols to avoid conflict with real FriBiDi library #5651 [nulano]

    8.3.1 (2021-07-06)

    • Catch OSError when checking if fp is sys.stdout #5585 [radarhere]

    • Handle removing orientation from alternate types of EXIF data #5584 [radarhere]

    • Make Image.array take optional dtype argument #5572 [t-vi, radarhere]

    8.3.0 (2021-07-01)

    • Use snprintf instead of sprintf. CVE-2021-34552 #5567 [radarhere]

    • Limit TIFF strip size when saving with LibTIFF #5514 [kmilos]

    • Allow ICNS save on all operating systems #4526 [baletu, radarhere, newpanjing, hugovk]

    • De-zigzag JPEG's DQT when loading; deprecate convert_dict_qtables #4989 [gofr, radarhere]

    • Replaced xml.etree.ElementTree #5565 [radarhere]

    ... (truncated)

    Commits
    • 8013f13 8.3.2 version bump
    • 23c7ca8 Update CHANGES.rst
    • 8450366 Update release notes
    • a0afe89 Update test case
    • 9e08eb8 Raise ValueError if color specifier is too long
    • bd5cf7d FLI tests for Oss-fuzz crash.
    • 94a0cf1 Fix 6-byte OOB read in FliDecode
    • cece64f Add 8.3.2 (2021-09-02) [CI skip]
    • e422386 Add release notes for Pillow 8.3.2
    • 08dcbb8 Pillow 8.3.2 supports Python 3.10 [ci skip]
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • How to train on custom dataset?

    How to train on custom dataset?

    Thanks for your great work and your efforts.

    My question is that is there a way to train a model on a custom dataset suppose I want to perform "MMIS" operation on my custom dataset where I have "trainA" "trainB" and want to transform trainA to trainB with any of the MMIS operations how to do that?

    another thing I want to know is if we can perform any other additional transformation like fog to the normal image or vice versa.

    Thank You

    opened by Mahmood-Hussain 1
  • parmeters

    parmeters

    I want to know why there is a big gap between the preset parameters of G and D. Is this a suitable default value in your experiment? When I set ngf and ndf to 32, the parameters of G are about 100M and D is 0.3M (I'm afraid it's too small). Thank you

    opened by fiftywu 1
  • Bump pillow from 8.1.1 to 8.2.0

    Bump pillow from 8.1.1 to 8.2.0

    Bumps pillow from 8.1.1 to 8.2.0.

    Release notes

    Sourced from pillow's releases.

    8.2.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.2.0.html

    Changes

    Dependencies

    Deprecations

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.2.0 (2021-04-01)

    • Added getxmp() method #5144 [UrielMaD, radarhere]

    • Add ImageShow support for GraphicsMagick #5349 [latosha-maltba, radarhere]

    • Do not load transparent pixels from subsequent GIF frames #5333 [zewt, radarhere]

    • Use LZW encoding when saving GIF images #5291 [raygard]

    • Set all transparent colors to be equal in quantize() #5282 [radarhere]

    • Allow PixelAccess to use Python int when parsing x and y #5206 [radarhere]

    • Removed Image._MODEINFO #5316 [radarhere]

    • Add preserve_tone option to autocontrast #5350 [elejke, radarhere]

    • Fixed linear_gradient and radial_gradient I and F modes #5274 [radarhere]

    • Add support for reading TIFFs with PlanarConfiguration=2 #5364 [kkopachev, wiredfool, nulano]

    • Deprecated categories #5351 [radarhere]

    • Do not premultiply alpha when resizing with Image.NEAREST resampling #5304 [nulano]

    • Dynamically link FriBiDi instead of Raqm #5062 [nulano]

    • Allow fewer PNG palette entries than the bit depth maximum when saving #5330 [radarhere]

    • Use duration from info dictionary when saving WebP #5338 [radarhere]

    • Stop flattening EXIF IFD into getexif() #4947 [radarhere, kkopachev]

    ... (truncated)

    Commits
    • e0e353c 8.2.0 version bump
    • ee635be Merge pull request #5377 from hugovk/security-and-release-notes
    • 694c84f Fix typo [ci skip]
    • 8febdad Review, typos and lint
    • fea4196 Reorder, roughly alphabetic
    • 496245a Fix BLP DOS -- CVE-2021-28678
    • 22e9bee Fix DOS in PSDImagePlugin -- CVE-2021-28675
    • ba65f0b Fix Memory DOS in ImageFont
    • bb6c11f Fix FLI DOS -- CVE-2021-28676
    • 5a5e6db Fix EPS DOS on _open -- CVE-2021-28677
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump pillow from 6.2.2 to 7.1.0

    Bump pillow from 6.2.2 to 7.1.0

    ⚠️ Dependabot is rebasing this PR ⚠️

    If you make any changes to it yourself then they will take precedence over the rebase.


    Bumps pillow from 6.2.2 to 7.1.0.

    Release notes

    Sourced from pillow's releases.

    7.1.0

    https://pillow.readthedocs.io/en/stable/releasenotes/7.1.0.html

    7.0.0

    https://pillow.readthedocs.io/en/stable/releasenotes/7.0.0.html

    Changelog

    Sourced from pillow's changelog.

    7.1.0 (2020-04-01)

    • Fix multiple OOB reads in FLI decoding #4503 [wiredfool]

    • Fix buffer overflow in SGI-RLE decoding #4504 [wiredfool, hugovk]

    • Fix bounds overflow in JPEG 2000 decoding #4505 [wiredfool]

    • Fix bounds overflow in PCX decoding #4506 [wiredfool]

    • Fix 2 buffer overflows in TIFF decoding #4507 [wiredfool]

    • Add APNG support #4243 [pmrowla, radarhere, hugovk]

    • ImageGrab.grab() for Linux with XCB #4260 [nulano, radarhere]

    • Added three new channel operations #4230 [dwastberg, radarhere]

    • Prevent masking of Image reduce method in Jpeg2KImagePlugin #4474 [radarhere, homm]

    • Added reading of earlier ImageMagick PNG EXIF data #4471 [radarhere]

    • Fixed endian handling for I;16 getextrema #4457 [radarhere]

    • Release buffer if function returns prematurely #4381 [radarhere]

    • Add JPEG comment to info dictionary #4455 [radarhere]

    • Fix size calculation of Image.thumbnail() #4404 [orlnub123]

    • Fixed stroke on FreeType < 2.9 #4401 [radarhere]

    • If present, only use alpha channel for bounding box #4454 [radarhere]

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • What is the network structure in the box?

    What is the network structure in the box?

    Hello, I have some questions about the paper. What is the network structure in the box(between the two Content ResBlk)? I didn't find the descriptions in your paper? And what you meaning of using Spectral Norm for all layers in the network? Is not Batch normalization and Instance normalization?

    opened by Gamkiu 1
  • Alpha is not working

    Alpha is not working

    Thanks for your nice work. If I change the alpha during the testing time it produces false results. It generates black images instead of stylized imaged. what could be the possible reasons?

    I trained the network with alpha 1.0 but I guess it should work if we change it during the inference time.

    thank you

    opened by Mahmood-Hussain 0
  • Reducing Generator size

    Reducing Generator size

    Thank you for sharing your research! I am training my own model with a dataset with pairs of images 256x256. The output Generator size is 1.59GB, is there a way to reduce the size of the model based on the size of the images? Or in any other way?

    Thank you for your advice.

    opened by cyprian 0
  • Other GPU ids throw error

    Other GPU ids throw error

    When using any other GPU devices ID, except 0, the code throws error. " Traceback (most recent call last): File "test.py", line 12, in opt = TestOptions().parse() File "/home/jang_sa/phd/AI/domain_adaptation/TSIT/options/base_options.py", line 178, in parse torch.cuda.set_device(opt.gpu_ids[0]) File "/home/jang_sa/Software/anaconda3/envs/tsit/lib/python3.7/site-packages/torch/cuda/init.py", line 263, in set_device torch._C._cuda_setDevice(device) RuntimeError: CUDA error: invalid device ordinal " The GPUs are available and device IDs are valid but still error is got !! any solution for this problem ?

    opened by sandeepjangir07 4
Owner
Liming Jiang
Ph.D. student, MMLab@NTU
Liming Jiang
Totally Versatile Miscellanea for Pytorch

Totally Versatile Miscellania for PyTorch Thomas Viehmann [email protected] This repository collects various things I have implmented for PyTorch Laye

Thomas Viehmann 428 Dec 28, 2022
The code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention.

CrossFormer This repository is the code for our paper CrossFormer: A Versatile Vision Transformer Based on Cross-scale Attention. Introduction Existin

cheerss 238 Jan 6, 2023
Versatile Generative Language Model

Versatile Generative Language Model This is the implementation of the paper: Exploring Versatile Generative Language Model Via Parameter-Efficient Tra

Zhaojiang Lin 17 Dec 2, 2022
Learning Versatile Neural Architectures by Propagating Network Codes

Learning Versatile Neural Architectures by Propagating Network Codes Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang,

Mingyu Ding 36 Dec 6, 2022
Exploring Versatile Prior for Human Motion via Motion Frequency Guidance (3DV2021)

Exploring Versatile Prior for Human Motion via Motion Frequency Guidance This is the codebase for video-based human motion reconstruction in human-mot

Jiachen Xu 5 Jul 14, 2022
PyTorch implementation of SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching

SMODICE: Versatile Offline Imitation Learning via State Occupancy Matching This is the official PyTorch implementation of SMODICE: Versatile Offline I

Jason Ma 14 Aug 30, 2022
[CVPR 2022 Oral] Versatile Multi-Modal Pre-Training for Human-Centric Perception

Versatile Multi-Modal Pre-Training for Human-Centric Perception Fangzhou Hong1  Liang Pan1  Zhongang Cai1,2,3  Ziwei Liu1* 1S-Lab, Nanyang Technologic

Fangzhou Hong 96 Jan 3, 2023
This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models are Pix2Pix, Pix2PixHD, CycleGAN and PointWise.

RGB2NIR_Experimental This repository contains several image-to-image translation models, whcih were tested for RGB to NIR image generation. The models

null 5 Jan 4, 2023
Framework for joint representation learning, evaluation through multimodal registration and comparison with image translation based approaches

CoMIR: Contrastive Multimodal Image Representation for Registration Framework ?? Registration of images in different modalities with Deep Learning ??

Methods for Image Data Analysis - MIDA 55 Dec 9, 2022
Image-to-Image Translation with Conditional Adversarial Networks (Pix2pix) implementation in keras

pix2pix-keras Pix2pix implementation in keras. Original paper: Image-to-Image Translation with Conditional Adversarial Networks (pix2pix) Paper Author

William Falcon 141 Dec 30, 2022
Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

null 364 Dec 14, 2022
Code for Dual Contrastive Learning for Unsupervised Image-to-Image Translation, NTIRE, CVPRW 2021.

arXiv Dual Contrastive Learning Adversarial Generative Networks (DCLGAN) We provide our PyTorch implementation of DCLGAN, which is a simple yet powerf

null 119 Dec 4, 2022
This is the PyTorch implementation of GANs N’ Roses: Stable, Controllable, Diverse Image to Image Translation

Official PyTorch repo for GAN's N' Roses. Diverse im2im and vid2vid selfie to anime translation.

null 1.1k Jan 1, 2023
Unsupervised Image-to-Image Translation

UNIT: UNsupervised Image-to-image Translation Networks Imaginaire Repository We have a reimplementation of the UNIT method that is more performant. It

Ming-Yu Liu 劉洺堉 1.9k Dec 26, 2022
PyTorch implementation of "Image-to-Image Translation Using Conditional Adversarial Networks".

pix2pix-pytorch PyTorch implementation of Image-to-Image Translation Using Conditional Adversarial Networks. Based on pix2pix by Phillip Isola et al.

mrzhu 383 Dec 17, 2022
Image-to-Image Translation in PyTorch

CycleGAN and pix2pix in PyTorch New: Please check out contrastive-unpaired-translation (CUT), our new unpaired image-to-image translation model that e

Jun-Yan Zhu 19k Jan 7, 2023
Breaking the Dilemma of Medical Image-to-image Translation

Breaking the Dilemma of Medical Image-to-image Translation Supervised Pix2Pix and unsupervised Cycle-consistency are two modes that dominate the field

Kid Liet 86 Dec 21, 2022
Maximum Spatial Perturbation for Image-to-Image Translation (Official Implementation)

MSPC for I2I This repository is by Yanwu Xu and contains the PyTorch source code to reproduce the experiments in our CVPR2022 paper Maximum Spatial Pe

null 51 Dec 14, 2022