Hello Hukai Wu.. The available pretrained model is saved as .npz file. During training for every epoch, separately for generator, discriminator and snapshot a .npz file is getting saved. How to save full trained model in .npz format?

I got the following error while loading the blending_GAN.npz for resuming the training. I want to fine tune the data on a different dataset. Its training on it from scratch but cannot load the provided weights

Traceback (most recent call last):
  File "train_blending_gan.py", line 178, in <module>
    main()
  File "train_blending_gan.py", line 170, in main
    chainer.serializers.load_npz(args.resume, trainer)
  File "/usr/local/lib/python3.6/dist-packages/chainer/serializers/npz.py", line 243, in load_npz
    d.load(obj)
  File "/usr/local/lib/python3.6/dist-packages/chainer/serializer.py", line 83, in load
    obj.serialize(self)
  File "/usr/local/lib/python3.6/dist-packages/chainer/training/trainer.py", line 388, in serialize
    self.updater.serialize(serializer['updater'])
  File "/usr/local/lib/python3.6/dist-packages/chainer/training/updaters/standard_updater.py", line 265, in serialize
    iterator.serialize(serializer['iterator:' + name])
  File "/usr/local/lib/python3.6/dist-packages/chainer/iterators/multiprocess_iterator.py", line 218, in serialize
    self.current_position)
  File "/usr/local/lib/python3.6/dist-packages/chainer/serializers/npz.py", line 176, in __call__
    dataset = self.npz[key]
  File "/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py", line 266, in __getitem__
    raise KeyError("%s is not a file in the archive" % key)
KeyError: 'updater/iterator:main/current_position is not a file in the archive'

Note : The weight file was renamed for convenience, the path is correct.

Hi,

Thanks for sharing your work.

I have run the latest code and got the test result (supervised GP-GAN) like the following image:

The result looks like it does not work properly. Is this an issue?

installed cupy 6.3.0 and chainer 6.3.0 as requested, getting the following error for running run_gp_gan.py:

File "cupy/core/core.pyx", line 2284, in cupy.core.core.tensordot_core File "cupy/core/core.pyx", line 2383, in cupy.core.core.tensordot_core File "cupy/cuda/cublas.pyx", line 927, in cupy.cuda.cublas.sgemmEx File "cupy/cuda/cublas.pyx", line 308, in cupy.cuda.cublas.check_status cupy.cuda.cublas.CUBLASError: CUBLAS_STATUS_EXECUTION_FAILED

List of modules installed:

chainer==6.3.0 cupy==6.3.0 cycler==0.10.0 decorator==4.4.2 fastrlock==0.5 filelock==3.0.12 imageio==2.9.0 kiwisolver==1.3.1 matplotlib==3.3.3 networkx==2.5 numpy==1.19.4 Pillow==8.0.1 pkg-resources==0.0.0 protobuf==3.7.1 pyparsing==2.4.7 python-dateutil==2.8.1 PyWavelets==1.1.1 scikit-image==0.15.0 scipy==1.5.4 six==1.15.0 typing==3.6.6 typing-extensions==3.6.6

Upon selecting 2 images of the same size I run into this error for the run_gp_gan.py. Can someone guide. What could be the possible error I am committing?

First of all, thanks for the great work which point out a framework for high resolution on GAN!!

But I don't understand what's the purpose of the functions "laplacian_param" and "gaussian_param" in gp_gan.py?

Could you give me some hint?

python run_gp_gan.py --image_size 640 --src_image im2.jpg --dst_image im1.jpg --mask_image mask_screenshot_05.09.2022.png --blended_image images/test_images/result.jpg
Input arguments: nef: 64 ngf: 64 nc: 3 nBottleneck: 4000 ndf: 64 image_size: 640 color_weight: 1 sigma: 0.5 gradient_kernel: normal smooth_sigma: 1 supervised: True nz: 100 n_iteration: 2 gpu: 0 g_path: models/blending_gan.npz unsupervised_path: models/unsupervised_blending_gan.npz list_path: result_folder: blending_result src_image: im2.jpg dst_image: im1.jpg mask_image: mask_screenshot_05.09.2022.png blended_image: images/test_images/result.jpg

I waited for so long and I didn't,t get output result

Successfully built the docker container. Running your test script (in docker) or doing import chainer in the python3 console results in following output. However importing other installed packages works (like numpy).

>>> import chainer Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python3.5/dist-packages/chainer/__init__.py", line 10, in <module> from chainer import backends # NOQA File "/usr/local/lib/python3.5/dist-packages/chainer/backends/__init__.py", line 1, in <module> from chainer.backends import cuda # NOQA File "/usr/local/lib/python3.5/dist-packages/chainer/backends/cuda.py", line 45, in <module> from chainer.backends import _cpu File "/usr/local/lib/python3.5/dist-packages/chainer/backends/_cpu.py", line 8, in <module> import chainerx File "/usr/local/lib/python3.5/dist-packages/chainerx/__init__.py", line 28, in <module> from chainerx.creation.from_data import asanyarray # NOQA ImportError: No module named 'chainerx.creation'

Following the installation for chainerX resulted in output that suggests that everything is properly installed.

Did I miss something? I executed following command in the docker image: python3 run_gp_gan.py --src_image images/test_images/src.jpg --dst_image images/test_images/dst.jpg --mask_image images/test_images/mask.png --blended_image images/test_images/result.png

How I used docker: Built the image with docker image build -t gp-gan:1.0 .Then I started the container via docker container run -dit --name gp-gan gp-gan:1.0 and then attached via docker attach gp-gan

Any help would be greatly appreciated! :)

Hi - thanks for making an implementation of your work available. Could you share the CLI input arguments that were used for creating the bear/beach example from Figure 10 please? Using the images from here, I haven't been able to obtain comparable results with the default parameter settings. I can reproduce the results in images/test_images correctly with the default params. Thanks in advance!

Hello! I was tried to look on the performance of pretrained model throw all available links (GitHub and Colab), but it seems the model weights are under private configuration at the moment. Can you help in accessing please?

Hi @wuhuikai , I have image with 600x799, and I initially just not modify the image size, and get following error: copy_paste_init = obj_im_pyramid[0] * mask_init + bg_im_pyramid[0] * (1 - mask_init) ValueError: operands could not be broadcast together with shapes (64,64,4) (64,64,3)

Then due to there is no separate width, height argument, I pass in --image_size=800, and the python execution hangs after it print out input arguments. Any idea what I did wrong?

When I use my own datasets, after I run the sheet "train_blending_gan.py", the command line seems to be stuck like this:

Input arguments:
        nef: 64
        ngf: 64
        nc: 3
        nBottleneck: 4000
        ndf: 64
        lr_d: 0.0002
        lr_g: 0.002
        beta1: 0.5
        l2_weight: 0.999
        gpu: 1
        n_epoch: 25
        data_root: steel_plates
        load_size: 64
        image_size: 64
        ratio: 0.5
        val_ratio: 0.05
        d_iters: 5
        clamp_lower: -0.01
        clamp_upper: 0.01
        experiment: encoder_decoder_blending_result
        test_folder: samples
        workers: 4
        batch_size: 64
        test_size: 64
        train_samples: 150000
        test_samples: 256
        manual_seed: 5
        resume: 
        snapshot_interval: 1
        print_interval: 1
        plot_interval: 10

Create & Init models ...
        Init G network ...
        Init D network ...
        Copy models to gpu 1 ...
Init models done ...

Load images from steel_plates ...
        21 folders in total, 1 val folders ...
        Trainset contains 150000 image files
        Valset contains 256 image files

Saving samples to encoder_decoder_blending_result/samples ...

And, when I kill this script, the command line shown like this:

^CTraceback (most recent call last):
  File "train_blending_gan.py", line 183, in <module>
    main()
  File "train_blending_gan.py", line 164, in main
    train_batch = [trainset[idx][0] for idx in range(args.test_size)]
  File "train_blending_gan.py", line 164, in <listcomp>
    train_batch = [trainset[idx][0] for idx in range(args.test_size)]
  File "/home/wyy/anaconda3/envs/python36/lib/python3.6/site-packages/chainer/dataset/dataset_mixin.py", line 67, in __getitem__
    return self.get_example(index)
  File "/ssd3/wyy/projects/GP-GAN/dataset.py", line 84, in get_example
    obj_croped = self._crop(obj, rw, rh, sx, sy)
  File "/ssd3/wyy/projects/GP-GAN/dataset.py", line 66, in _crop
    im = resize(im, (rw, rh), order=1, preserve_range=False, mode='constant')
  File "/home/wyy/anaconda3/envs/python36/lib/python3.6/site-packages/skimage/transform/_warps.py", line 148, in resize
    cval=cval, mode=ndi_mode)
  File "/home/wyy/anaconda3/envs/python36/lib/python3.6/site-packages/scipy/ndimage/filters.py", line 299, in gaussian_filter
    mode, cval, truncate)
  File "/home/wyy/anaconda3/envs/python36/lib/python3.6/site-packages/scipy/ndimage/filters.py", line 217, in gaussian_filter1d
    return correlate1d(input, weights, axis, output, mode, cval, 0)
  File "/home/wyy/anaconda3/envs/python36/lib/python3.6/site-packages/scipy/ndimage/filters.py", line 95, in correlate1d
    origin)
KeyboardInterrupt

Is the size of the image in my dataset (5000x3000) too large? Or is it something else? I look forward to your reply and would appreciate it.