Official PyTorch implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

Hi, thanks for this incredible net. I've already trained the dataset, but when I aim to test it and I input the following command: python main.py --phrase test --light true However, I got the error in the log:

##### Information #####
# light :  True
# dataset :  selfie2anime
# batch_size :  1
# iteration per epoch :  1000000

##### Generator #####
# residual blocks :  4

##### Discriminator #####
# discriminator layer :  6

##### Weight #####
# adv_weight :  1
# cycle_weight :  10
# identity_weight :  10
# cam_weight :  1000
 [*] Load FAILURE
 [*] Test finished!

Hi, when I execute python main.py --dataset selfie2anime ,it tips RuntimeError:

number of epochs must be larger than or equal to one

##### Information #####
# light :  False
# dataset :  selfie2anime
# batch_size :  1
# iteration per epoch :  1000000

##### Generator #####
# residual blocks :  4

##### Discriminator #####
# discriminator layer :  6

##### Weight #####
# adv_weight :  1
# cycle_weight :  10
# identity_weight :  10
# cam_weight :  1000
Traceback (most recent call last):
  File "main.py", line 83, in <module>
    main()
  File "main.py", line 72, in main
    gan.build_model()
  File "H:\workspace\UGATIT-pytorch\UGATIT.py", line 100, in build_model
    self.trainA = ImageFolder(os.path.join('dataset', self.dataset, 'trainA'), train_transform)
  File "H:\workspace\UGATIT-pytorch\dataset.py", line 107, in __init__
    target_transform=target_transform)
  File "H:\workspace\UGATIT-pytorch\dataset.py", line 47, in __init__
    "Supported extensions are: " + ",".join(extensions)))
RuntimeError: Found 0 files in subfolders of: dataset\selfie2anime\trainA
Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif

but this folder hava default female_2321.jpg file

Hi, thank you for sharing your code with pytorch, and I find some importance trouble in your code. Maybe cause failed to reimplements model performance!!

In UGATIT.py -- train function:

1. when you update D , you do not use

self.genA2B.eval(), self.genB2A.eval() and in your code here

fake_B2A.detach(), fake_B2A.detach() ..... but only fake_B2A, fake_B2A. and so on...

so, when you want to only update D, but meanwhile, the G is also updated. It is the same error in update G

you can reference SPADE code , if I am wrong ,please tell me

    self.genA2B = ResnetGenerator(input_nc=3, output_nc=3, ngf=self.ch, n_blocks=self.n_res, img_size=self.img_size, light=self.light).to(self.device)
    self.genB2A = ResnetGenerator(input_nc=3, output_nc=3, ngf=self.ch, n_blocks=self.n_res, img_size=self.img_size, light=self.light).to(self.device)
    self.disGA = Discriminator(input_nc=3, ndf=self.ch, n_layers=7).to(self.device)
    self.disGB = Discriminator(input_nc=3, ndf=self.ch, n_layers=7).to(self.device)
    self.disLA = Discriminator(input_nc=3, ndf=self.ch, n_layers=5).to(self.device)
    self.disLB = Discriminator(input_nc=3, ndf=self.ch, n_layers=5).to(self.device)

I've never write python code.So how can I run it? I try to run python main.py --dataset selfie2anime in cmd.But some errors be displayed: Traceback (most recent call last): File "main.py", line 1, in <module> from UGATIT import UGATIT File "/Users/yh/Personal/UGATIT-pytorch/UGATIT.py", line 2, in <module> from dataset import ImageFolder File "/Users/yh/Personal/UGATIT-pytorch/dataset.py", line 1, in <module> import torch.utils.data as data ImportError: No module named torch.utils.data

run: python main.py --dataset my_dataset --light True error: RuntimeError: CUDA error: out of memory

I had sucessfully installed pytorch with GPU on computer.

import torch torch.cuda.is_available() True

This works well,and I find no error when running other pytorch codes.

Hi, I have noticed that the parameter Rho of AdaILN and ILN layer in pytorch version of UGATIT is not clipped or constrainted compared to your TF version and the function in paper.

Hi, in the paper, I find $L_{cam}^{D_{t}}$ have a log function, but in source code, only use MSE(UGATIT.py 182-189 lines). So, the paper adds unnecessarily log？

It seems like your implementation of AdaILN/ILN layer in pytorch significantly differs from tf version (actually, tf version is the correct one).

in_mean, in_var = torch.mean(torch.mean(input, dim=2, keepdim=True), dim=3, keepdim=True), torch.var(torch.var(input, dim=2, keepdim=True), dim=3, keepdim=True)
ln_mean, ln_var = torch.mean(torch.mean(torch.mean(input, dim=1, keepdim=True), dim=2, keepdim=True), dim=3, keepdim=True), torch.var(torch.var(torch.var(input, dim=1, keepdim=True), dim=2, keepdim=True), dim=3, keepdim=True)

This block implies that when you are trying to evaluate variance of each tensor in the batch over appropriate dimension, instead of variation over axes [2, 3] in IN and axes [1, 2, 3] in LN you evaluate variation of variation in IN and variation of variation of variation in LN which is wrong

Hi, thank you for sharing source code.

But, compared to tf version, ResNetAdaILNBlock layer doesn't seem to have a shortcut connection which adds the input of layer to output directory.

Hi, I found that the pytorch version is more unstable compared to tensorflow version of UGATIT. One difference I noticed in this version is the weight-decay is not used in the pytorch optimizer compared to TF version.

Fix to issue #23 (CUDA out of memory at 1000th epoch) as suggested by @07hyx06. Surrounded print output with torch.no_grad().

Tested by running on an internal large dataset. Before the fix, it stopped at 1000th epoch, complaining that it was out of memory. After fix, it's still running at 128,000 epochs. Memory initially 22157MB on GPU, and has not increased during training.

According to the appendix section B.2 in the paper, the weights should be initialized from a zero-centered normal distribution with a standard deviation of 0.02. But I did not find this part in the repo. Could anybody explain?

When calling python main.py --dataset maps --light True --phase test the model seems to fail to load the checkpoint weights. Any Idea what could be the problem?

Last output:

...

##### Weight #####
# adv_weight :  1
# cycle_weight :  10
# identity_weight :  10
# cam_weight :  1000
 [*] Load FAILURE
 [*] Test finished!
All done!

My folder structure:

├── dataset
│   ├── maps
│   │   ├── testA
│   │   ├── testB
│   │   ├── trainA
│   │   ├── trainB
│   │   └── valB