The example configuration files (https://github.com/nv-tlabs/datasetGAN_release/tree/master/datasetGAN/experiments), ex. "cat_16.json" or "car_20.json", contain a field named "average_latent" and "annotation_image_latent_path". Both of these fields are paths to .npy files.

This repo does not describe how these files are generated. Even after one trains a StyleGAN model, one cannot then use DatasetGAN without these files. Please provide documentation on how to create this file for a custom dataset.

I imagine that you'd want to provide examples using your up-to-date repos, ex. https://github.com/NVlabs/stylegan2-ada-pytorch and/or https://github.com/NVlabs/stylegan2-ada.

Hi, To run datasetgan, i need not only the model represented by .pt but also the npy binaries for average_latent and annotation_image_latent. However, the readme only has a configuration that works based on four pre-prepared config files. I understand that I need to write my own config and prepare the necessary data, but there is no explanation for that. Or is this code just a demo and not supposed to work with any data sets? Thank you.

Hello,

Thanks for your great work, I just wanted to know how can we use the stylegan2 pre-trained models for training the interpreter? do you think you will support it in the future?

I have trained my models with stylegan2-ada-pytorch from https://github.com/NVlabs/stylegan2-ada-pytorch

I got a cuda OOM while running train_interpreter, I found a variable "batch_size" in train_interpreter.py but changing it doesn't seem to help, can I limit the memory usage?(For example, from the json config) I have a 2080ti board with 11GB of GMEM. Thank you.

Opt {'exp_dir': 'model_dir/face_34', 'category': 'face', 'debug': False, 'dim': [512, 512, 5088], 'deeplab_res': 512, 'number_class': 34, 'testing_data_number_class': 34, 'max_training': 16, 'stylegan_ver': '1', 'annotation_data_from_w': False, 'annotation_mask_path': './dataset_release/annotation/training_data/face_processed', 'testing_path': './dataset_release/annotation/testing_data/face_34_class', 'average_latent': './dataset_release/training_latent/face_34/avg_latent_stylegan1.npy', 'annotation_image_latent_path': './dataset_release/training_latent/face_34/latent_stylegan1.npy', 'stylegan_checkpoint': './dataset_release/stylegan_pretrain/karras2019stylegan-ffhq-1024x1024.for_g_all.pt', 'model_num': 10, 'upsample_mode': 'bilinear'} /home/udemegane/anaconda3/envs/dataset/lib/python3.7/site-packages/torch/nn/functional.py:2973: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details. "See the documentation of nn.Upsample for details.".format(mode)) Traceback (most recent call last): File "train_interpreter.py", line 578, in <module> main(opts) File "train_interpreter.py", line 446, in main all_feature_maps_train_all, all_mask_train_all, num_data = prepare_data(args, palette) File "train_interpreter.py", line 403, in prepare_data return_upsampled_layers=True, use_style_latents=args['annotation_data_from_w']) File "../utils/utils.py", line 94, in latent_to_image affine_layers_upsamples = torch.FloatTensor(1, number_feautre, dim, dim).cuda() RuntimeError: CUDA out of memory. Tried to allocate 4.97 GiB (GPU 0; 10.76 GiB total capacity; 7.61 GiB already allocated; 2.33 GiB free; 7.65 GiB reserved in total by PyTorch)

1. In class generate_data， What is the purpose of using 10 models?
2. I noticed make_ training_ Data.py seems to be used to generate data， but I want to train the first stage with my own labeled training data. How should I operate?

Thank you！

At present, I use 256 * 256*3 images for training, and the final styleGAN output is 256 * 256 * 4864. Due to the limitation of memory, I can only load the features of about 100 images at a time under 400G memory, and it will take dozens of minutes. Is there any way to use more images for training? For example, is it feasible to reduce the number of input features？

Hi, Nice work! I am curious about how to construct the ADE-Car-12 and PASCAL-Car-5. In the paper, it says ADE-Car-20 contains 50 and 250 images for training and testing. I wonder what are those exact images (e.g. image index). Are they preprocessed? Is there any instruction for constructing it? I also wonder which 900 images are taken from Pascal Part for PASCAL-Car-5 and how are the 5 classes defined. Thanks very much!

Can you please provide greater clarity on where to download the styleGAN checkpoints from?

The linked repo has pickled models but not checkpoints and certainly not .pt checkpoints. Also its a tensorflow implementation so I'm a little confused...

Specfically where can I find:

• karras2019stylegan-cars-512x384.for_g_all.pt
• karras2019stylegan-cats-256x256.for_g_all.pt
• karras2019stylegan-celebahq-1024x1024.for_g_all.pt

Hello, Thank you so much for providing the code for this paper :) I am trying to generate an annotated face dataset, but I am having some trouble with the train_deeplab.py. In ln[95] there is all_pickle = glob.glob(data_path + '/*.pickle'), which loads the 16 manually annotated data if I understood the docs correctly. However, I can't seem to find these .pickle files. I guess one can load the training/test data provided in dataset_release, but it would still be missing the uncertainty_score.

Hi,

After having trained my own StyleGAN on images of 128x128 pixels in order to then use the DatasetGAN, the code of "train_interpreter.py" indicates that the total number of features that will go into the pixel_classifier (the value dim [-1]) would be of 79,691,776 - which is way too big and just not normal compared to the values presented in the configuration files (.json files).

Is there a way that would allow me to reduce this value or correct this ?

Note: This values of 79,691,776 (= 128x128x4864) was the one giving by the line of code "feature_maps = feature_maps.reshape(-1, args["dim"][2])" in the function "prepare_data"

Hi!

Thanks once again for your responsiveness :)

May I ask about the procedure used to train deeplab-v3 on 1024 images, I noticed the provided code in the repo samples train_interpreter.py, and trains train_deeplab.py on images with 512 x 512 resolution for face_34 task.

When doing the same for 1024 x 1024 images, the cross_validation script results in this:

Could you provide more information on the procedure for training deeplab on 1024 x 1024 images? How large was the training set that you used? The number of epochs, and batch size? I ask the latter, as I also had some issues with OOM cuda error, when training on 32G NVIDIA Tesla V100.

Hi @arieling,

Great job with this repository, it is awesome.

On the ReadMe page you state that:

"One can cache the data returned from prepare_data function to disk but it will increase trianing time due to I/O burden."

How would I implement this?

Thank You!

What is the exact PCK used for keypoint detection?

To be more specific, PCK means the keypoint accuracy where a keypoint is defined as "correct" if it lies in a range of the GT keypoint. I cannot find the exact range definition in the paper.

Thanks in advance.

Is it possible to use only the DatasetGan part in the new version under EditGan? I want to generate annotated dataset based on StyleGan 2 but without the app of EditGan, it is still possible?

Thank you for your code. I got this error: RuntimeError: mat1 and mat2 shapes cannot be multiplied (1x18 and 512x1024)

To fix this I add transpose on the latent variable before latent_to_image in make_training_data.py

The author indicated that the graphics card device was Tesla V100, which is relatively high configuration requirements for some general lab. I would like to ask if the author has performed calculations on other relatively low-profile graphics cards, such as RTX30 series or RTX20 series graphics cards?

Thank you for open-sourcing this project! We are trying to implement the model, and are trying to better understand the affine_layers variable that is used to generate the synthetic images. What are these layers that are being taken from the pre-trained StyleGAN1 (SG1)? Are they specific hidden layers? Also, if we were to swap this SG1 for our own model, would the corresponding affine_layers be sufficient along with the randomly generated latent variable?

Thank you for your time, we appreciate it!