hi,thanks for job. my win10 pc run stylegan3 is ok but run anycost-gan , demo.py need cuda, using force-native=1, my pc cant understand,anyone can help me,thanks very much.

import torch
import numpy as np
import os
from PIL import Image
from models.dynamic_channel import set_uniform_channel_ratio, reset_generator
import models


class FaceEditor:
    def __init__(self, config, device, anycost_resolution=1024, n_style_to_change=12):
        # load assets
        self.device = device
        self.anycost_channel = 1.0
        self.anycost_resolution = anycost_resolution
        self.n_style_to_change = n_style_to_change

        # build the generator
        self.generator = models.get_pretrained('generator', config).to(device)
        self.generator.eval()
        set_uniform_channel_ratio(self.generator, 0.5)  # set channel
        self.generator.target_res = anycost_resolution  # set resolution
        # self.generator.target_res = self.anycost_resolution
        self.mean_latent = self.generator.mean_style(10000)

        # select only a subset of the directions to use
        '''
        possible keys:
        ['00_5_o_Clock_Shadow', '01_Arched_Eyebrows', '02_Attractive', '03_Bags_Under_Eyes', '04_Bald', '05_Bangs',
            '06_Big_Lips', '07_Big_Nose', '08_Black_Hair', '09_Blond_Hair', '10_Blurry', '11_Brown_Hair', '12_Bushy_Eyebrows',
            '13_Chubby', '14_Double_Chin', '15_Eyeglasses', '16_Goatee', '17_Gray_Hair', '18_Heavy_Makeup', '19_High_Cheekbones',
            '20_Male', '21_Mouth_Slightly_Open', '22_Mustache', '23_Narrow_Eyes', '24_No_Beard', '25_Oval_Face', '26_Pale_Skin',
            '27_Pointy_Nose', '28_Receding_Hairline', '29_Rosy_Cheeks', '30_Sideburns', '31_Smiling', '32_Straight_Hair',
            '33_Wavy_Hair', '34_Wearing_Earrings', '35_Wearing_Hat', '36_Wearing_Lipstick', '37_Wearing_Necklace',
            '38_Wearing_Necktie', '39_Young']
        '''

        direction_map = {
            'smiling': '31_Smiling',
            'young': '39_Young',
            'wavy hair': '33_Wavy_Hair',
            'gray hair': '17_Gray_Hair',
            'blonde hair': '09_Blond_Hair',
            'eyeglass': '15_Eyeglasses',
            'mustache': '22_Mustache',
        }

        boundaries = models.get_pretrained('boundary', config)
        self.direction_dict = dict()
        for k, v in boundaries.items():
            self.direction_dict[k] = v.view(1, 1, -1)

    def get_latent_code(self, latent_code_path):
        latent_code = torch.from_numpy(np.load(os.path.join(latent_code_path))).view(1, -1, 512)
        return latent_code

    def get_direction_dict(self, attr_weights):
        final_dict = {}
        for key, value in attr_weights.items():
            if value == 0:
                continue
            final_dict[key] = value * self.direction_dict[key]
        return final_dict

    def get_boundary_dict(self):
        return self.direction_dict

    def generate_image(self, save_path, input_kwargs):
        def image_to_np(x):
            assert x.shape[0] == 1
            x = x.squeeze(0).permute(1, 2, 0)
            x = (x + 1) * 0.5  # 0-1
            x = (x * 255).cpu().numpy().astype('uint8')
            return x

        with torch.no_grad():
            out = self.generator(**input_kwargs)[0].clamp(-1, 1)
            out = image_to_np(out)
            out = np.ascontiguousarray(out)
            img_pil = Image.fromarray(out)
            img_pil.save(save_path)

    def edit(self, latent_code_path, attr_sliders, force_full_g=False):
        latent_code = torch.from_numpy(np.load(os.path.join(latent_code_path))).view(1, -1, 512).to(self.device)
        # input kwargs for the generator

        edited_code = latent_code.clone()
        for direction_name in attr_sliders.keys():
            edited_code[:, :self.n_style_to_change] = edited_code[:, :self.n_style_to_change] \
                                                 + attr_sliders[direction_name] * self.direction_dict[
                                                     direction_name].to(self.device)

        edited_code = edited_code.to(self.device)
        if not force_full_g:
            set_uniform_channel_ratio(self.generator, self.anycost_channel)
            self.generator.target_res = self.anycost_resolution
        return latent_code, edited_code

if __name__ == '__main__':
    gan_config = 'stylegan2-ffhq-config-f'
    fe = FaceEditor(config=gan_config, device='cuda:0')
    latent_code = torch.from_numpy(np.load(os.path.join(latent_code_path))).view(1, -1, 512).to(self.device)
    ori_kwargs = {'styles': ori, 'noise': None, 'randomize_noise': False, 'input_is_style': True}
      
    fe.generate_image(save_path=ori_save_path, input_kwargs=ori_kwargs)

image generate by config anycost-ffhq-config-f is pretty fine, but there the image generate with config stylegan2-ffhq-config-f is wrong. How can I fix the bug? Thankyou

Hello! Thanks for creating this.

I am trying to use this tool with another model (the LSUN Churches dataset) with sliders that represent attributes.

As I understand it, these are the steps I need to take to configure this toolset to work with a different dataset + pre-trained network:

• Make sure the format of the LSUN Churches dataset matches as described for other models

• Change the config name as described here to config_name = ''stylegan2-church-config-f", referring to the pre-trained network found here

• run models.get_pretrained('attribute-predictor') as described in the pre-trained-models section of README

• Change the relevant attribute labels in the files that show up during this search

I am just wondering if there any obvious steps I am missing to get this working, I am very new to the world of GANs and toolsets. Thank you for your time 😊

import torch
import numpy as np
import os, random
from PIL import Image
from tqdm import tqdm
from models.dynamic_channel import set_uniform_channel_ratio, reset_generator
import models
import time
import cv2
config = 'anycost-ffhq-config-f'
device = 'cuda:2'

class Face_Editor():
    def __init__(self):
        self.init_model()

    def init_model(self):
        self.anycost_channel = 1.0
        self.anycost_resolution = 1024
        self.generator = models.get_pretrained('generator', config).to(device)
        self.generator.eval()

    def sample(self):
        torch.manual_seed(1601)
        # latent = torch.randn(1, 1, 512, device=device)
        # mean_style = self.generator.mean_style(10000)
        # self.input_kwargs = {'styles':latent, 'return_rgbs':True, 'truncation':0.5,
        #                      'truncation_style':mean_style, 'randomize_noise':False}
        # style = torch.randn(1, 18, 512, device=device)
        style = np.load('/simple/zlp1/masters/anycost-gan/assets/demo_ori/projected_latents/00_ryan.npy')
        style = torch.from_numpy(style).view(1, -1, 512).to(device)
        self.input_kwargs = {'styles': style,
                            'noise': None, 'randomize_noise': False, 'input_is_style': True}
        image = self.generate_image()
        image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
        cv2.imshow('image', image)
        cv2.waitKey(0)

    def generate_image(self):
        def image_to_np(x):
            assert x.shape[0] == 1
            x = x.squeeze(0).permute(1, 2, 0)
            x = (x + 1) * 0.5  # 0-1
            x = (x * 255).cpu().numpy().astype('uint8')
            return x

        with torch.no_grad():
            print(self.input_kwargs)
            out = self.generator(**self.input_kwargs)[0].clamp(-1, 1)
            out = image_to_np(out)
            return out


if __name__ == '__main__':
    FE = Face_Editor()
    FE.sample()

Dear anycost-gan team,

Thank you for sharing this great work, I really like it.

Would you minding sharing the intermedia checkpoint for mult-resolution step? To train anycost gan, we need to do 3 steps:

1. Training the original StyleGAN2 on FFHQ
2. Training Anycost GAN: mult-resolution
3. Training Anycost GAN: adaptive-channel

You provide the checkpoint for 1st and 3rd steps. Would you minding aslo sharing the checkpoint for the second step? I understand that I can train it by myself, but 8 gpus for 5 days is really too heave resource for us.

Thank you for your help.

Best Wishes,

Alex

Hi @junyanz @songhan , I found a possible bug, not sure, if not please correct me, thank you. when you calculate the fid, you use the transform with random flip: https://github.com/mit-han-lab/anycost-gan/blob/master/tools/calc_inception.py#L53 but when training code, there is just clamp, no flip: https://github.com/mit-han-lab/anycost-gan/blob/master/tools/train_gan.py#L279 That maybe lead to wrong evaluation result

Hi, @tonylins Thank you for your good paper.

In the case of this Github, only 256x256 resolution can be encoded. However, it seems that only the resolutions of 1024x1024 and 512x512 are uploaded through the decoder.

What I want to test is to encode and decode a 256x256 image and check whether the same image as the original image comes out.

Can you send me 256x256 anycost-ffhq decoder weight?

https://github.com/mit-han-lab/anycost-gan/blob/19229bdc525ecd00ca7c2322b192504325bda9e0/models/anycost_gan.py#L85 here 'z' is "cpu" device while self.style needs "cuda" tensor??? Error is: RuntimeError: input must be a CUDA tensor i figured out the reason is the code then running to FusedLeakyReLUFunction ,which requires CUDA tensor. But in demo.py, the device is "cpu". Any idea how to fix this problem?

Dear mit-han-lab,

Thank you for sharing with us this great work, I really like it.

In Table 1, you show that multiple resolution outputs have higher image quality compared to single resolution training in config E. Have you try config F, which is the standard stylegan2 mode?

According to FFHQ 1024 leadboard, the stylegan2 has FID of 2.84, while anycost GAN has FID of 2.99, which is a little bit worse. So I am wondering if you use config F as standard StyleGAN2, will you get better results than standard StyleGAN2?

Thank you for your help.

Best Wishes,

Alex

Question 1: how to generate latent image code in custom image editing Question 2: when customizing image editing properties, can i use all 40 properties to modify without needing to be retrained?I see that demo.py uses eight properties

您好，感谢分享。 有一些不太理解的地方，希望能解答。

encoder, generator, discriminator的训练流程是怎样的？ 我猜测是 先discriminator, generator训练完成后， 使用generator来训练encoder。这种流程，encoder是不会影响generator。 那么是否可以三个模型一起训练。互相影响，达到最优。

Hi, thanks so much for this awesome library. Congrats on the great work!

I've a question regarding the color of the generated image using the stylegan2-ffhq-config-f.

Why are they yellowish and have less contrast? I'm thinking it might be the difference in how the training images for the different generators are normalized, is there a way to reverse the normalization after the images are generated using the stylegan2-ffhq-config-f?

I tried to play around with the parameters in the image_to_np function, but it did not work.

Please see attached for a sample.

Please let me know what I did wrong here or if this is an improvement that could be made.

Thank you!

Hi @tonylins , I tried with adaptive-channel training, and used the teacher model from your dropbox: https://www.dropbox.com/sh/l8g9amoduz99kjh/AAAY9LYZk2CnsO43ywDrLZpEa?dl=0 stylegan2-ffhq-config-f.pt

But got this error: Could you please kindly tell me what's problem/cause of it, thank you.

Hello, great work! I am wondering what options you use to calculate the projected latents in assets/demo/projected_latents? I am trying to recreate them using the default parameters via: python3 tools/project.py 00_ryan.jpg But the resulting vectors are numerically different and, when viewed in demo.py: (1) the projected image is good but clearly different than the projected image preloaded in the repo and (2) the editing directions don't seem to work very well for this set of latent codes.

Below I've included a screenshot of the behavior I am seeing. Note the differences in his neckline from the demo projection and the lack of any meaningful change in the output image.