PyTorch Implementation of AnimeGANv2

Quick question: Is it possible to train with new style like from Netflix style animation "ARCANE" I really love their rendering of face.

If possible, is it hard, does it take a long time using M1?

Could you let us know who to train another model like "Face Portrait v1 "?

as I know, Animegan2 is not for facial style transfer.. so I really want to know the detail steps to train facial model by using Animegan2.

Thanks very much!

I added some details in the readme:

Torch Hub Usage

You can load Animegan v2 via torch.hub:

import torch
model = torch.hub.load('xhlulu/animegan2-pytorch', 'generator').eval()
# convert your image into tensor here
out = model(img_tensor)

You can load with various configs (more details in the torch docs):

model = torch.hub.load(
    "xhlulu/animegan2-pytorch",
    "generator",
    pretrained=True, # or give URL to a pretrained model
    device="cuda", # or "cpu" if you don't have a GPU
    progress=True, # show progress
)

Currently, the following pretrained shorthands are available:

model = torch.hub.load("xhlulu/animegan2-pytorch", "generator", pretrained="celeba_distill")
model = torch.hub.load("xhlulu/animegan2-pytorch", "generator", pretrained="face_paint_512_v1")
model = torch.hub.load("xhlulu/animegan2-pytorch", "generator", pretrained="face_paint_512_v2")
model = torch.hub.load("xhlulu/animegan2-pytorch", "generator", pretrained="paprika")

You can also load the face2paint util function. First, install dependencies:

pip install torchvision Pillow numpy

Then, import the function using torch.hub:

face2paint = torch.hub.load(
    'xhlulu/animegan2-pytorch', 'face2paint', 
    size=512, device="cpu"
)

img = Image.open(...).convert("RGB")
out = face2paint(model, img)

Would probably need to update the download links to bryandlee/animegan2-pytorch before merging

Hello, when I use the [face_paint_512_v2.pt] model to convert the full body photo, it is very ugly, but the photo containing only the face is good, how can I combine the face only and the full body photo together? Does the model need to be retrained?

like this

photo1 use other app,i think its use animegan2. photo2 convert full body photo.its ugly width face photo3 only convert face, its nice.and face like photo 1

The demo on HG Spaces is amazing! I used it for a few photos and really like it.

Unfortunately it seems it's getting quite crowded and takes over 20min to process due to the queue. Also it runs on a CPU.

I added a colab notebook that lets you upload a photo and run the model. It works on both CPU and GPUs. You can see what it looks like here: https://colab.research.google.com/github/xhlulu/animegan2-pytorch/blob/main/colab_demo.ipynb

Hi Bryan, I wanted to make this to work locally on my machine, so I don't need to queue, but I kept getting error when running: python convert_weights.py:

Traceback (most recent call last):
  File "/Users/jimmygunawan/animegan2-pytorch-main-2/convert_weights.py", line 7, in <module>
    from AnimeGANv2.net import generator as tf_generator
  File "/Users/jimmygunawan/animegan2-pytorch-main-2/AnimeGANv2/net/generator.py", line 1, in <module>
    import tensorflow.contrib as tf_contrib
ModuleNotFoundError: No module named 'tensorflow.contrib'

I am not programmer, so far I did try to install tensorflow via miniforge on this Mac machine, I think I also have conda stuff and tensorflow installed, but issue above persist.

Any idea how to fix this? Thanks!

In the basic usage, you say that Weight Conversion from the Original Repo (Requires TensorFlow 1.x) TensorFlow 1.x can only installed for python2. However the code in test.py is using python3 syntax. How did you manage to make it work?

I'm interested in experimenting your Paprika style model, using [GitHub] repo for converting images. Usually, people use this interface, which takes so long time.

I want to train the offered available pre-trained models on Google Colab Notebook and experiment Additional Model Weights, but not only I couldn't manage to run Torch Hub Usage according to instructions but also I couldn't run the face2paint option due to the following error:

---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
<ipython-input-13-35ffb2517470> in <module>()
----> 1 face2paint = torch.hub.load('bryandlee/animegan2-pytorch:main', 'face2paint', size=512, device="cpu")
      2 
      3 img = Image.open(...).convert("RGB")
      4 out = face2paint(model, img)

8 frames
/usr/lib/python3.7/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs)
    647 class HTTPDefaultErrorHandler(BaseHandler):
    648     def http_error_default(self, req, fp, code, msg, hdrs):
--> 649         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    650 
    651 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 403: rate limit exceeded 

I can't figure out to resolve this issue to run the model on the Colab notebook. Please feel free to edit or comment on Google Colab Notebook I shared.

my torch.version is 1.2.0 and have problem about torch.load() . What version do you use?

RuntimeError: ./face_paint_512_v2.pt is a zip archive (did you mean to use torch.jit.load()?)

Great work! I'd like to convert your pretrained model, face_paint_512_v2, to pytorch mobile.

from torch.utils.mobile_optimizer import optimize_for_mobile

Do you know the model specifics that will allow a conversion to the lite interpreter for mobile?

Hello, I want to implement the training code for this project.

• Did you learn the same as the original animeganv2?

• Did you have any other additions for training code implementation?

When I try the project passing an image I get this error : I have version 1.13.0 for torch and 0.14.0 for torchvision

env: « wine »: No file or folder of this type

Great work! I'd like to convert your pretrained model, celeba_distill, to pytorch mobile. from torch.utils.mobile_optimizer import optimize_for_mobile Do you know the model specifics that will allow a conversion to the lite interpreter for mobile?

I was wondering ıf it would be possible to re use the data that is generated (probably) while tracking the head it one place to put it in it's original place.

use case: after processing a video with animegan2 the head would be in its own place so the original head can be replaced with the animegan version easliy.

right now I can do it by tracking the original head and using the tracking data to put the processed head in place but it is too jumpy...

some outputs that your model Face Portrait v2 generated have a very good anime outline : and

How to do it ? I tried to redo training it but failed !

Hello, your work is really great. I try to apply the filter to some full-body photos. But in some pictures, they look dirty in some parts. I find another app that may use your work, their results look much better, and the pictures are 'clean'. Do you have any ideas to improve the results? I uploaded the results and the original picture I used.