UI2I via StyleGAN2 - Unsupervised image-to-image translation method via pre-trained StyleGAN2 network

Hi @HideUnderBush, thanks for your great work, and when I try to use it, I encounter some problems. When I use python closed_from _factorization.py, I got aRuntimeError: no default TensorFlow session found. please call dnnlib.init_tf(), then I do like it says, call dnnlin.init_tf() before tf.get_default_session(), but got no module error as above. Hope you can help, thanks~

Hi just stumbled across this and it looks great, particularly the anime generation images. Looks like you're essentially using the method I described in some of my blog posts around transfer learning, using one latent code from one model in another and layer swapping. (https://.www.justinpinkney.com) and I'm glad to see you cite Doron and I in for our Toonify work!

We actually have a paper on arxiv that descirbes this approach, particularly focussing on the idea of layer swapping you're using It would be really great if you could cite our actual paper: Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains

Perhaps as prior work where you are describing the "layer swapping" you perform?

Hi!

According to your paper, it takes about a second to invert an image to its latent representation: "...and another 0.8 − 1 s for the inversion process". However, in your current implementation it is an iterative optimization process, which takes more than a minute with the default settings. Could you, please, clarify how do you successfully accomplish the inversion task within 1 second?

Traceback (most recent call last): File "convert_weight.py", line 235, in with open(args.path, "rb") as f: PermissionError: [Errno 13] Permission denied: 'D:/converted'

I set up environment correctly in conda, python 3.6 and tf=1.14 torch with cpp extension and after defeating all the errors i got this one. Any idea what this might be? :) thanks ia

@HideUnderBush, I got this error when runclosed_form_factorization.py

Setting up TensorFlow plugin "fused_bias_act.cu": Preprocessing... Loading... Done. Setting up TensorFlow plugin "upfirdn_2d.cu": Preprocessing... Loading... Done. Traceback (most recent call last): File "closed_form_factorization.py", line 14, in ckpt = torch.load(args.ckpt, map_location='cuda:0') File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 595, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 766, in _legacy_load raise RuntimeError("Invalid magic number; corrupt file?") RuntimeError: Invalid magic number; corrupt file?

BUG: Line 104 and Line 107 should add parameter input_is_latent=True or the content and the reference cannot be properly used.

ISSUE: Also truncation=0.5 can sometimes be too strict, making the output's content and style not matched with the input content image and the reference image.

https://github.com/HideUnderBush/UI2I_via_StyleGAN2/blob/bd4cd6af326f22f55c58b9b3886d1a5bbdb7460f/closed_form_factorization.py#L17

I’ve been digging through GitHub for help on g_ema tweaking from generator

I have this ticket - https://github.com/danielroich/PTI/issues/26

The maths is a bit beyond me / but I suspect I need to update g_ema like you’ve done here. I need to play around with this repo to investigate further.

Hi, @HideUnderBush! Thanks for you amazing works! I try to reimplement the face2anime experiments on Danbooru Datasets. However, I face some confusions, could you give me some advice? Step 1: According to your scripts, I use the 512 px stylegan2 ckpt pretrained on ffhq datasets as base, and finetune on Danbooru Datasets. (I didn't change any other params, is that right?) Step 2: I use closed_form_factorization.py to decompose the model has trained 35000 iterations (35000.pt) to get factor.out file. Step 3: I try to achieve image inversion (size is 512), however, when the optimization program finished, I got an almost black result. The MSE loss is very large. (The loss is about 1.4-1.7). Are there any key points I forgot? I wish you can point out some mistakes about my steps. Thanks for your jobs!

Thank you for your amazing work. I am a little confused about the layer swap part in your implementation. It seems that you first pass the latent code into the base model and then extract the intermediate results for the target model as the following.

img1, swap_res = g_ema1([input_latent], input_is_latent=True, save_for_swap=True, swap_layer=args.swap_layer)

for i in range(args.stylenum):
    sample_z_style = torch.randn(1, 512, device=args.device)
    img_style, _ = g_ema2([input_latent], truncation=0.5, truncation_latent=mean_latent, swap=True, swap_layer=args.swap_layer,  swap_tensor=swap_res, multi_style=True, multi_style_latent=[sample_z_style])
    print(i)
    img_style_name = args.output + "_style_" + str(i) + ".png"
    img_style = make_image(img_style)
    out_style = Image.fromarray(img_style[0])
    out_style.save(img_style_name)```

Is it true that you are trying to keep the low level information such as shape and pose from original model and put the lightening and texture from the target model?