hitting drive limits on downloading models, another solution other than pydrive is weights could be under project release

see torch hub

https://pytorch.org/docs/stable/hub.html

"Pretrained weights can either be stored locally in the github repo, or loadable by torch.hub.load_state_dict_from_url(). If less than 2GB, it’s recommended to attach it to a project release and use the url from the release. In the example above torchvision.models.resnet.resnet18 handles pretrained, alternatively you can put the following logic in the entrypoint definition."

and a similar example from animegan hubconf, although the weights are much smaller in size here

https://github.com/bryandlee/animegan2-pytorch/blob/main/hubconf.py

or the models can be hosted on huggingface, see

https://huggingface.co/models

I am confused by this error. I have uploaded my own style images and used the supplied iu.jpeg file to transform. I get this error in the last cell. I have verified that the style images are 3 channel images.

How many style images are needed? Does the format of the images matter?

Hi, I have some questions about the number of finetune data pairs. According to stylize.ipynb's part: Finetune StyleGAN, I find the variable "random_alpha" is not be used. If use only one reference style image, then I only have one pair to finetune the styleGAN？Could you plz tell me what am I doing wrong? Thanks a lot.

for the finetuning stage in colab, wandb is useful to track metrics, see https://github.com/danielroich/PTI#weights-and-biases for example. This would be helpful when running multiple experiments in colab, if you have time, otherwise I can also look into this as a PR. Thanks

dlib is very slow to build, possible to use a alternative like from here https://github.com/happy-jihye/FFHQ-Alignment, for example https://github.com/happy-jihye/FFHQ-Alignment/blob/master/FFHQ-Alignmnet/ffhq-align.py

more example usage here: https://github.com/happy-jihye/FFHQ-Alignment/blob/master/FFHQ-Alignmnet/FFHQ-Alignment.ipynb

Hi,

I'm playing with your notebook (awesome work btw!) and I try to give it another pretrained GAN from Awesome Pretrained StyleGAN2.

I used the anime one (PyTorch implementation from here) but I get

TypeError                                 Traceback (most recent call last)

[<ipython-input-10-2395bdddca96>](https://localhost:8080/#) in <module>()
     22 
     23 #print(ckpt)
---> 24 generator.load_state_dict(ckpt["g"], strict=False)
     25 
     26 #@title Generate results

TypeError: 'Generator' object is not subscriptable

Do I need further operation on the model to make it compatible?

Thank you

Hi!

Thanks for sharing this awesome work :-)

I'm wondering on the difference in perceptual image quality when using the LPIPS model (as stated in the paper) vs. the StyleGAN discriminator (as used in the updated collab notebook) for the perceptual loss. In your experience, what kind of difference does using the StyleGAN discriminator have on the image quality, when compared to using LPIPS?

Hey @mchong6! 👋

I really enjoyed playing with JoJoGAN. This pull request makes it possible to run your model inside a Docker environment, which makes it easier for other people to run it. We're using an open source tool called Cog to make this process easier.

This also means we can make a web page where other people can try out your model! View it here: https://replicate.ai/mchong6/jojogan

Claim your page here so you can edit it, and we'll feature it on our website and tweet about it too. We have added some examples to the page, but do claim the page so you can own the page, customize the Example gallery as you like, and push any future updates to the web demo.

In case you're wondering who I am, I'm from Replicate, where we're trying to make machine learning reproducible. We got frustrated that we couldn't run all the really interesting ML work being done. So, we're going round implementing models we like. 😊

Hi, thank you for sharing source code. I can't find how many paired image of dataset C. From your experiment, at least how many paired (wi, y) can have a good result?

when device is set to cpu in colab and hardware accelerator is none

No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

IndexError Traceback (most recent call last) in () 24 from tqdm import tqdm 25 import lpips ---> 26 from model import * 27 from e4e_projection import projection as e4e_projection 28 from restyle_projection import projection as restyle_projection

7 frames /usr/local/lib/python3.7/dist-packages/torch/utils/cpp_extension.py in _get_cuda_arch_flags(cflags) 1604 arch_list.append(arch) 1605 arch_list = sorted(arch_list) -> 1606 arch_list[-1] += '+PTX' 1607 else: 1608 # Deal with lists that are ' ' separated (only deal with ';' after)

IndexError: list index out of range

I took a picture with a face in it and generated an image using the align_face function. Then, using the same picture, I generated another image by manual cropping as mentioned in https://github.com/mchong6/JoJoGAN/issues/18#issuecomment-1008996040. When I pass both of these through the e4e_projection function and view the final image, the results are very different although the images which were passed are very similar. Do you have any idea of why this might be?

The new version of Cog improves the Python API, along with several other changes. Particularly pydantic is now used for Predictor and the previous version will be deprecated.

This PR upgrades the Replicate demo and API to Cog version >= 0.1. I have already pushed this to Replicate, so you don't need to do anything for the demo to keep working :) https://replicate.com/mchong6/jojogan

Currently, I keep getting CUDA run out of memory error if I use more than 4 reference image with 16GB GPU. Is there a way to train with more images? I'd like the model to be more general.

Hello. Thank you for providing a greate code.

I have a issue on running prediction.

if i predice on cuda:1, the inference is not working.....

i trace the code step by step and I found "op/fused_act.py" load fused_bias_Act.cpp and fused_bias_act_kernel.cu. that cpp code cannot another gpu...

how can i predict with other gpu?.....

Hey, great work! I had a couple of queries:

1. In the paper it is mentioned that there are 26 style modulation layers, but in the code it seems to be 18 as n_latent = 18.
2. What exactly do the style parameters s(w) correspond to in the code?
3. For the pertained styles, is there any way to control low-level features like eyes, nose, etc. without training again? I know that while fine-tuning we can use blending (using RIS) and different masks for controlling them but is there any way for a model which is already fine-tuned?
4. I see a change in results when fine-tuning the model using JoJo's photo. Is it because of e4e being used instead of ReStyle in the code?