[CVPR 2020] GAN Compression: Efficient Architectures for Interactive Conditional GANs

Overview

GAN Compression

project | paper | videos | slides

[NEW!] GAN Compression is accepted by T-PAMI! We released our T-PAMI version in the arXiv v4!

[NEW!] We release the codes of our interactive demo and include the TVM tuned model. It achieves 8FPS on Jetson Nano GPU now!

[NEW!] Add support to the MUNIT, a multimodal unsupervised image-to-image translation approach! Please follow the test commands to test the pre-trained models and the tutorial to train your own models!

teaser We introduce GAN Compression, a general-purpose method for compressing conditional GANs. Our method reduces the computation of widely-used conditional GAN models, including pix2pix, CycleGAN, and GauGAN, by 9-21x while preserving the visual fidelity. Our method is effective for a wide range of generator architectures, learning objectives, and both paired and unpaired settings.

GAN Compression: Efficient Architectures for Interactive Conditional GANs
Muyang Li, Ji Lin, Yaoyao Ding, Zhijian Liu, Jun-Yan Zhu, and Song Han
MIT, Adobe Research, SJTU
In CVPR 2020.

Demos

Overview

overviewGAN Compression framework: ① Given a pre-trained teacher generator G', we distill a smaller “once-for-all” student generator G that contains all possible channel numbers through weight sharing. We choose different channel numbers for the student generator G at each training step. ② We then extract many sub-generators from the “once-for-all” generator and evaluate their performance. No retraining is needed, which is the advantage of the “once-for-all” generator. ③ Finally, we choose the best sub-generator given the compression ratio target and performance target (FID or mIoU), perform fine-tuning, and obtain the final compressed model.

Performance

performance

GAN Compression reduces the computation of pix2pix, cycleGAN and GauGAN by 9-21x, and model size by 4.6-33x.

Colab Notebook

PyTorch Colab notebook: CycleGAN and pix2pix.

Prerequisites

  • Linux
  • Python 3
  • CPU or NVIDIA GPU + CUDA CuDNN

Getting Started

Installation

  • Clone this repo:

    git clone [email protected]:mit-han-lab/gan-compression.git
    cd gan-compression
  • Install PyTorch 1.4 and other dependencies (e.g., torchvision).

    • For pip users, please type the command pip install -r requirements.txt.
    • For Conda users, we provide an installation script scripts/conda_deps.sh. Alternatively, you can create a new Conda environment using conda env create -f environment.yml.

CycleGAN

Setup

  • Download the CycleGAN dataset (e.g., horse2zebra).

    bash datasets/download_cyclegan_dataset.sh horse2zebra
  • Get the statistical information for the ground-truth images for your dataset to compute FID. We provide pre-prepared real statistic information for several datasets. For example,

    bash datasets/download_real_stat.sh horse2zebra A
    bash datasets/download_real_stat.sh horse2zebra B

Apply a Pre-trained Model

  • Download the pre-trained models.

    python scripts/download_model.py --model cycle_gan --task horse2zebra --stage full
    python scripts/download_model.py --model cycle_gan --task horse2zebra --stage compressed
  • Test the original full model.

    bash scripts/cycle_gan/horse2zebra/test_full.sh
  • Test the compressed model.

    bash scripts/cycle_gan/horse2zebra/test_compressed.sh
  • Measure the latency of the two models.

    bash scripts/cycle_gan/horse2zebra/latency_full.sh
    bash scripts/cycle_gan/horse2zebra/latency_compressed.sh
  • There may be a little differences between the results of above models and those of the paper since we retrained the models. We also release the compressed models of the paper. If there are such inconsistencies, you could try the following commands to test our paper models:

    python scripts/download_model.py --model cycle_gan --task horse2zebra --stage legacy
    bash scripts/cycle_gan/horse2zebra/test_legacy.sh
    bash scripts/cycle_gan/horse2zebra/latency_legacy.sh

Pix2pix

Setup

  • Download the pix2pix dataset (e.g., edges2shoes).

    bash datasets/download_pix2pix_dataset.sh edges2shoes-r
  • Get the statistical information for the ground-truth images for your dataset to compute FID. We provide pre-prepared real statistics for several datasets. For example,

    bash datasets/download_real_stat.sh edges2shoes-r B
    bash datasets/download_real_stat.sh edges2shoes-r subtrain_B

Apply a Pre-trained Model

  • Download the pre-trained models.

    python scripts/download_model.py --model pix2pix --task edges2shoes-r --stage full
    python scripts/download_model.py --model pix2pix --task edges2shoes-r --stage compressed
  • Test the original full model.

    bash scripts/pix2pix/edges2shoes-r/test_full.sh
  • Test the compressed model.

    bash scripts/pix2pix/edges2shoes-r/test_compressed.sh
  • Measure the latency of the two models.

    bash scripts/pix2pix/edges2shoes-r/latency_full.sh
    bash scripts/pix2pix/edges2shoes-r/latency_compressed.sh
  • There may be a little differences between the results of above models and those of the paper since we retrained the models. We also release the compressed models of the paper. If there are such inconsistencies, you could try the following commands to test our paper models:

    python scripts/download_model.py --model pix2pix --task edges2shoes-r --stage legacy
    bash scripts/pix2pix/edges2shoes-r/test_legacy.sh
    bash scripts/pix2pix/edges2shoes-r/latency_legacy.sh

GauGAN

Setup

  • Prepare the cityscapes dataset. Check here for preparing the cityscapes dataset.

  • Get the statistical information for the ground-truth images for your dataset to compute FID. We provide pre-prepared real statistics for several datasets. For example,

    bash datasets/download_real_stat.sh cityscapes A

Apply a Pre-trained Model

  • Download the pre-trained models.

    python scripts/download_model.py --model gaugan --task cityscapes --stage full
    python scripts/download_model.py --model gaugan --task cityscapes --stage compressed
  • Test the original full model.

    bash scripts/gaugan/cityscapes/test_full.sh
  • Test the compressed model.

    bash scripts/gaugan/cityscapes/test_compressed.sh
  • Measure the latency of the two models.

    bash scripts/gaugan/cityscapes/latency_full.sh
    bash scripts/gaugan/cityscapes/latency_compressed.sh
  • There may be a little differences between the results of above models and those of the paper since we retrained the models. We also release the compressed models of the paper. If there are such inconsistencies, you could try the following commands to test our paper models:

    python scripts/download_model.py --model gaugan --task cityscapes --stage legacy
    bash scripts/gaugan/cityscapes/test_legacy.sh
    bash scripts/gaugan/cityscapes/latency_legacy.sh

MUNIT

Setup

  • Prepare the dataset (e.g., edges2shoes-r).

    bash datasets/download_pix2pix_dataset.sh edges2shoes-r
    python datasets/separate_A_and_B.py --input_dir database/edges2shoes-r --output_dir database/edges2shoes-r-unaligned
    python datasets/separate_A_and_B.py --input_dir database/edges2shoes-r --output_dir database/edges2shoes-r-unaligned --phase val
  • Get the statistical information for the ground-truth images for your dataset to compute FID. We provide pre-prepared real statistics for several datasets. For example,

    bash datasets/download_real_stat.sh edges2shoes-r B
    bash datasets/download_real_stat.sh edges2shoes-r-unaligned subtrain_B

Apply a Pretrained Model

  • Download the pre-trained models.

    python scripts/download_model.py --model gaugan --task cityscapes --stage full
    python scripts/download_model.py --model gaugan --task cityscapes --stage compressed
  • Test the original full model.

    bash scripts/munit/edges2shoes-r_fast/test_full.sh
  • Test the compressed model.

    bash scripts/munit/edges2shoes-r_fast/test_compressed.sh
  • Measure the latency of the two models.

    bash scripts/munit/edges2shoes-r_fast/latency_full.sh
    bash scripts/munit/edges2shoes-r_fast/latency_compressed.sh

Cityscapes Dataset

For the Cityscapes dataset, we cannot provide it due to license issue. Please download the dataset from https://cityscapes-dataset.com and use the script prepare_cityscapes_dataset.py to preprocess it. You need to download gtFine_trainvaltest.zip and leftImg8bit_trainvaltest.zip and unzip them in the same folder. For example, you may put gtFine and leftImg8bit in database/cityscapes-origin. You need to prepare the dataset with the following commands:

python datasets/get_trainIds.py database/cityscapes-origin/gtFine/
python datasets/prepare_cityscapes_dataset.py \
--gtFine_dir database/cityscapes-origin/gtFine \
--leftImg8bit_dir database/cityscapes-origin/leftImg8bit \
--output_dir database/cityscapes \
--train_table_path datasets/train_table.txt \
--val_table_path datasets/val_table.txt

You will get a preprocessed dataset in database/cityscapes and a mapping table (used to compute mIoU) in dataset/table.txt.

To support mIoU computation, you need to download a pre-trained DRN model drn-d-105_ms_cityscapes.pth from http://go.yf.io/drn-cityscapes-models. By default, we put the drn model in the root directory of the repo. Then you can test our compressed models on cityscapes after you have downloaded our models.

COCO-Stuff Dataset

We follow the same COCO-Stuff dataset preparation as NVlabs/spade. Specifically, you need to download train2017.zip, val2017.zip, stuffthingmaps_trainval2017.zip, and annotations_trainval2017.zip from nightrome/cocostuff. The images, labels, and instance maps should be arranged in the same directory structure as in datasets/coco_stuff. In particular, we used an instance map that combines both the boundaries of "things instance map" and "stuff label map". To do this, we used a simple script datasets/coco_generate_instance_map.py.

To support mIoU computation, you need to download a pre-trained DeeplabV2 model deeplabv2_resnet101_msc-cocostuff164k-100000.pth and also put it in the root directory of the repo.

Performance of Released Models

Here we show the performance of all our released models:

Model Dataset Method #Parameters MACs Metric
FID mIoU
CycleGAN horse→zebra Original 11.4M 56.8G 65.75 --
GAN Compression (Paper) 0.342M 2.67G 65.33 --
GAN Compression (Retrained) 0.357M 2.55G 65.12 --
Fast GAN Compression 0.355M 2.64G 65.19 --
Pix2pix edges→shoes Original 11.4M 56.8G 24.12 --
GAN Compression (Paper) 0.700M 4.81G 26.60 --
GAN Compression (Retrained) 0.822M 4.99G 26.70 --
Fast GAN Compression 0.703M 4.83G 25.76 --
Cityscapes Original 11.4M 56.8G -- 42.06
GAN Compression (Paper) 0.707M 5.66G -- 40.77
GAN Compression (Retrained) 0.781M 5.59G -- 38.63
Fast GAN Compression 0.867M 5.61G -- 41.71
map→arial photo
Original 11.4M 56.8G 47.91 --
GAN Compression 0.746M 4.68G 48.02 --
Fast GAN Compression 0.708M 4.53G 48.67 --
GauGAN Cityscapes Original 93.0M 281G 57.60 61.04
GAN Compression (Paper) 20.4M 31.7G 55.19 61.22
GAN Compression (Retrained) 21.0M 31.2G 56.43 60.29
Fast GAN Compression 20.2M 31.3G 56.25 61.17
COCO-Stuff Original 97.5M 191G 21.38 38.78
Fast GAN Compression 26.0M 35.5G 25.06 35.05
MUNIT edges→shoes Original 15.0M 77.3G 30.13 --
Fast GAN Compression 1.10M 2.63G 30.53 --

Training

Please refer to the tutorial of Fast GAN Compression and GAN Compression on how to train models on our datasets and your own.

FID Computation

To compute the FID score, you need to get some statistical information from the groud-truth images of your dataset. We provide a script get_real_stat.py to extract statistical information. For example, for the edges2shoes dataset, you could run the following command:

python get_real_stat.py \
--dataroot database/edges2shoes-r \
--output_path real_stat/edges2shoes-r_B.npz \
--direction AtoB

For paired image-to-image translation (pix2pix and GauGAN), we calculate the FID between generated test images to real test images. For unpaired image-to-image translation (CycleGAN), we calculate the FID between generated test images to real training+test images. This allows us to use more images for a stable FID evaluation, as done in previous unconditional GANs research. The difference of the two protocols is small. The FID of our compressed CycleGAN model increases by 4 when using real test images instead of real training+test images.

Code Structure

To help users better understand and use our code, we briefly overview the functionality and implementation of each package and each module.

Citation

If you use this code for your research, please cite our paper.

@inproceedings{li2020gan,
  title={GAN Compression: Efficient Architectures for Interactive Conditional GANs},
  author={Li, Muyang and Lin, Ji and Ding, Yaoyao and Liu, Zhijian and Zhu, Jun-Yan and Han, Song},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2020}
}

Acknowledgements

Our code is developed based on pytorch-CycleGAN-and-pix2pix, SPADE, and MUNIT.

We also thank pytorch-fid for FID computation, drn for cityscapes mIoU computation and deeplabv2 for Coco-Stuff mIoU computation.

Comments
  • get_real_stat.py error

    get_real_stat.py error

    @lmxyy i am trying everything from scratch, when running get_real_stat.py i get an error. why does it require creating a val folder? there is already valA and valB

    (gan) home@home-lnx:~/programs/level 2/gan-compression$ python get_real_stat.py --dataroot database/horse2zebra/ --output_path real_stat/horse2zebra_B.npz --direction AtoB
    Traceback (most recent call last):
      File "get_real_stat.py", line 84, in <module>
        main(opt)
      File "get_real_stat.py", line 15, in main
        dataloader = create_dataloader(opt)
      File "/home/home/programs/level 2/gan-compression/data/__init__.py", line 45, in create_dataloader
        dataloader = CustomDatasetDataLoader(opt, verbose)
      File "/home/home/programs/level 2/gan-compression/data/__init__.py", line 97, in __init__
        self.dataset = dataset_class(opt)
      File "/home/home/programs/level 2/gan-compression/data/aligned_dataset.py", line 24, in __init__
        self.AB_paths = sorted(make_dataset(self.dir_AB))  # get image paths
      File "/home/home/programs/level 2/gan-compression/data/image_folder.py", line 45, in make_dataset
        assert os.path.isdir(dir) or os.path.islink(dir), '%s is not a valid directory' % dir
    AssertionError: database/horse2zebra/val is not a valid directory
    
    (gan) home@home-lnx:~/programs/level 2/gan-compression/database/horse2zebra$ tree -d
    .
    ├── testA
    ├── testB
    ├── trainA
    ├── trainB
    ├── valA -> testA
    └── valB -> testB
    
    opened by ghost 23
  • Searching is slow.How can I continue searching when it is break.

    Searching is slow.How can I continue searching when it is break.

    I just have one GPU. Searching is so slow. I want to run another code. How can I continue searching after finish another project?

    65%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 37494/57600 [40:38:20<137:38:38, 24.65s/it]MACs: 4.364G Params: 1.987M {'config_str': '48_32_32_48_40_32_24_16', 'macs': 4363780096, 'fid': 68.67807846983607} 65%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 37495/57600 [40:38:48<143:48:26, 25.75s/it]MACs: 5.282G Params: 1.987M {'config_str': '48_32_32_48_40_32_16_64', 'macs': 5282070528, 'fid': 64.41984751671731} 65%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 37496/57600 [40:39:19<152:29:44, 27.31s/it]MACs: 4.825G Params: 1.987M

    Thank you!

    opened by Feywell 7
  • My Own Full model  apply problem

    My Own Full model apply problem

    l have a pretrained full cyclegan model from the project "junyanz/pytorch-CycleGAN-and-pix2pix", but when l run bash scripts/cycle_gan/loldataset/test_full.sh RuntimeError: Error(s) in loading state_dict for ResnetGenerator: Missing key(s) in state_dict: "model.10.conv_block.6.weight", "model.10.conv_block.6.bias", "model.11.conv_block.6.weight", "model.11.conv_block.6.bias", "model.12.conv_block.6.weight", "model.12.conv_block.6.bias", "model.13.conv_block.6.weight", "model.13.conv_block.6.bias", "model.14.conv_block.6.weight", "model.14.conv_block.6.bias", "model.15.conv_block.6.weight", "model.15.conv_block.6.bias", "model.16.conv_block.6.weight", "model.16.conv_block.6.bias", "model.17.conv_block.6.weight", "model.17.conv_block.6.bias", "model.18.conv_block.6.weight", "model.18.conv_block.6.bias". Unexpected key(s) in state_dict: "model.10.conv_block.5.weight", "model.10.conv_block.5.bias", "model.11.conv_block.5.weight", "model.11.conv_block.5.bias", "model.12.conv_block.5.weight", "model.12.conv_block.5.bias", "model.13.conv_block.5.weight", "model.13.conv_block.5.bias", "model.14.conv_block.5.weight", "model.14.conv_block.5.bias", "model.15.conv_block.5.weight", "model.15.conv_block.5.bias", "model.16.conv_block.5.weight", "model.16.conv_block.5.bias", "model.17.conv_block.5.weight", "model.17.conv_block.5.bias", "model.18.conv_block.5.weight", "model.18.conv_block.5.bias". Do you know how to solve it?

    opened by qiaozhuocheng 6
  • Why it is getting slower and slower when training the supernets?

    Why it is getting slower and slower when training the supernets?

    I try to compress pix2pixhd. But I find it will cost longer time in one iter. For example, it cost 0.4s per images in the first epoch. But it will cost 8s per image in the 5 epoch. Do you have the same question?

    opened by Feywell 6
  • ValueError: Required crop size is larger than input image size

    ValueError: Required crop size is larger than input image size

    I am training a MobileNet-style teacher model on my own dataset, the input image size is 1084*708, there is my running command: python train.py --dataroot database/endoscolor --model cycle_gan --log_dir logs/cycle_gan/endoscolor/mobile --real_stat_A_path real_stat/endoscolor_A.npz --real_stat_B_path real_stat/endoscolor_B.npz --gpu_ids 0 --preprocess scale_width_and_crop --load_size 1084 --crop_size 360 I have a trained model from "junyanz/pytorch-CycleGAN-and-pix2pix", and I used the same command:" --preprocess scale_width_and_crop --load_size 1084 --crop_size 360", but I got VlueRrror at epoch8:

    **Traceback (most recent call last):
    File "train.py", line 5, in
    trainer.start()
    File "/home/pengchengh/Codes/gan-compression-master/trainer.py", line 95, in start 'Saving the latest model (epoch %d, total_iters %d)' % (epoch, total_iter)) File "/home/pengchengh/Codes/gan-compression-master/trainer.py", line 63, in evaluate metrics = self.model.evaluate_model(iter)
    File "/home/pengchengh/Codes/gan-compression-master/models/cycle_gan_model.py", line 282, in evaluate_model for i, data_i in enumerate(tqdm(eval_dataloader, desc='Eval %s ' % direction, position=2, leave=False)): File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/tqdm/std.py", line 1107, in iter for obj in iterable:
    File "/home/pengchengh/Codes/gan-compression-master/data/init.py", line 113, in iter for i, data in enumerate(self.dataloader):
    File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 435, in next data = self._next_data()
    File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1085, in _next_data return self._process_data(data) File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1111, in _process_data data.reraise()
    File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/_utils.py", line 428, in reraise raise self.exc_type(msg)
    ValueError: Caught ValueError in DataLoader worker process 0. Original Traceback (most recent call last):
    File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 198, in _worker_loop data = fetcher.fetch(index) File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index]
    File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/pengchengh/Codes/gan-compression-master/data/single_dataset.py", line 36, in getitem A = self.transform(A_img)
    File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 67, in call img = t(img)
    File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, kwargs) File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 585, in forward i, j, h, w = self.get_params(img, self.size) File "/home/pengchengh/anaconda3/envs/pytorch/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 542, in get_params "Required crop size {} is larger then input image size {}".format((th, tw), (h, w)) ValueError: Required crop size (360, 360) is larger then input image size (235, 360)

    opened by PatrickHaopc 5
  • real_stats for COCO

    real_stats for COCO

    Could you share the real_stats for the COCO data? It is not downloadable, and running the get_real_stat.py file for the coco data returns error: python get_real_stat.py --dataroot database/coco_stuff --output_path real_stat/coco_A.npz --direction BtoA --dataset_mode coco

    RuntimeError: Sizes of tensors must match except in dimension 0. Got 424 and 640 in dimension 2 (The offending index is 1)

    Thanks.

    opened by deJQK 5
  • There is a bug in tensor2im

    There is a bug in tensor2im

    The normlize is not work, when tensor.dim() == 4.

        if image_tensor.dim() == 4:
            # transform each image in the batch
            images_np = []
            for b in range(image_tensor.size(0)):
                one_image = image_tensor[b]
                one_image_np = tensor2im(one_image)
                images_np.append(one_image_np.reshape(1, *one_image_np.shape))
            images_np = np.concatenate(images_np, axis=0)
    

    https://github.com/mit-han-lab/gan-compression/blob/master/utils/util.py#L69

    opened by Feywell 5
  • Is Pre-distillation a required step?

    Is Pre-distillation a required step?

    I want to know the purpose of pre-distillation in the GAN-compression pipeline. How does it improve the pruning pipeline? It is not mentioned anywhere in the paper.

    opened by pkhigh 5
  • Question about distillation

    Question about distillation

    take CycleGAN compression as an example.

    the teacher generator: a MobileNet-based CycleGAN trained from scratch the student generator: a MobileNet-based distillation CycleGAN model the final generator: a fine-tuned sub network of student generator

    Is my understanding correct?

    Which means the original CycleGAN model(with normal conv) is not needed in compression algorthm?

    opened by 07hyx06 5
  • TypeError: _output_padding() missing 1 required positional argument: 'num_spatial_dims'

    TypeError: _output_padding() missing 1 required positional argument: 'num_spatial_dims'

    when I run the script train_supernet.sh in the fast gan compression,it occurs an error " TypeError: _output_padding() missing 1 required positional argument: 'num_spatial_dims'". image and the task script is "bash scripts/cycle_gan/horse2zebra_fast/train_supernet.sh"

    opened by dianxin556 4
  • Distill Problem

    Distill Problem

    I tried to train a pix2pix model on the edges2shoes-r dataset using train_full.sh.

    #!/usr/bin/env bash
    python distill.py --dataroot database/edges2shoes-r \
      --distiller resnet \
      --log_dir logs/pix2pix/edges2shoes-r/distill \
      --batch_size 4 \
      --restore_teacher_G_path logs/pix2pix/edges2shoes-r/train/checkpoints/latest_net_G.pth \
      --restore_pretrained_G_path logs/pix2pix/edges2shoes-r/train/checkpoints/latest_net_G.pth \
      --pretrained_netG resnet_9blocks \
      --teacher_netG resnet_9blocks \
      --student_netG resnet_9blocks \
      --restore_D_path logs/pix2pix/edges2shoes-r/train/checkpoints/latest_net_D.pth \
      --real_stat_path real_stat/edges2shoes-r_B.npz \
      --meta_path datasets/metas/edges2shoes-r/train1.meta 
    

    After training, I used this bash, but I get an AssertionError. In weight_transfer.py line 14, in transfer_Conv2d assert isinstance(m1, nn.Conv2d) and isinstance(m2, (nn.Conv2d, SuperConv2d)) How can I solve this problem?

    opened by saijo0404 4
  • [Question] About SuperSeparableConv2d

    [Question] About SuperSeparableConv2d

    Hi, thanks for sharing this great work. In the regular Cyclegan Resnet block, both Conv2d use the same number channels, usually 64*4. In the SuperMobileResnetBlock, the first SuperSeparableConv2d uses the sampled number of channels from the resnet_configs channels-64-cycleGAN, and the second uses the input.size(1). I would expect both SuperSeparableConv2d to use the same number of channels as in the original generator. I would appreciate your help understanding this, Thank you.

    opened by saduf 2
  • Bump certifi from 2019.11.28 to 2022.12.7

    Bump certifi from 2019.11.28 to 2022.12.7

    Bumps certifi from 2019.11.28 to 2022.12.7.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump pillow from 7.0.0 to 9.3.0

    Bump pillow from 7.0.0 to 9.3.0

    Bumps pillow from 7.0.0 to 9.3.0.

    Release notes

    Sourced from pillow's releases.

    9.3.0

    https://pillow.readthedocs.io/en/stable/releasenotes/9.3.0.html

    Changes

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    9.3.0 (2022-10-29)

    • Limit SAMPLESPERPIXEL to avoid runtime DOS #6700 [wiredfool]

    • Initialize libtiff buffer when saving #6699 [radarhere]

    • Inline fname2char to fix memory leak #6329 [nulano]

    • Fix memory leaks related to text features #6330 [nulano]

    • Use double quotes for version check on old CPython on Windows #6695 [hugovk]

    • Remove backup implementation of Round for Windows platforms #6693 [cgohlke]

    • Fixed set_variation_by_name offset #6445 [radarhere]

    • Fix malloc in _imagingft.c:font_setvaraxes #6690 [cgohlke]

    • Release Python GIL when converting images using matrix operations #6418 [hmaarrfk]

    • Added ExifTags enums #6630 [radarhere]

    • Do not modify previous frame when calculating delta in PNG #6683 [radarhere]

    • Added support for reading BMP images with RLE4 compression #6674 [npjg, radarhere]

    • Decode JPEG compressed BLP1 data in original mode #6678 [radarhere]

    • Added GPS TIFF tag info #6661 [radarhere]

    • Added conversion between RGB/RGBA/RGBX and LAB #6647 [radarhere]

    • Do not attempt normalization if mode is already normal #6644 [radarhere]

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • fix(sec): upgrade numpy to 1.22.2

    fix(sec): upgrade numpy to 1.22.2

    What happened?

    There are 1 security vulnerabilities found in numpy 1.18.1

    What did I do?

    Upgrade numpy from 1.18.1 to 1.22.2 for vulnerability fix

    What did you expect to happen?

    Ideally, no insecure libs should be used.

    The specification of the pull request

    PR Specification from OSCS

    opened by 645775992 0
  • Bump protobuf from 3.11.3 to 3.18.3

    Bump protobuf from 3.11.3 to 3.18.3

    Bumps protobuf from 3.11.3 to 3.18.3.

    Release notes

    Sourced from protobuf's releases.

    Protocol Buffers v3.18.3

    C++

    Protocol Buffers v3.16.1

    Java

    • Improve performance characteristics of UnknownFieldSet parsing (#9371)

    Protocol Buffers v3.18.2

    Java

    • Improve performance characteristics of UnknownFieldSet parsing (#9371)

    Protocol Buffers v3.18.1

    Python

    • Update setup.py to reflect that we now require at least Python 3.5 (#8989)
    • Performance fix for DynamicMessage: force GetRaw() to be inlined (#9023)

    Ruby

    • Update ruby_generator.cc to allow proto2 imports in proto3 (#9003)

    Protocol Buffers v3.18.0

    C++

    • Fix warnings raised by clang 11 (#8664)
    • Make StringPiece constructible from std::string_view (#8707)
    • Add missing capability attributes for LLVM 12 (#8714)
    • Stop using std::iterator (deprecated in C++17). (#8741)
    • Move field_access_listener from libprotobuf-lite to libprotobuf (#8775)
    • Fix #7047 Safely handle setlocale (#8735)
    • Remove deprecated version of SetTotalBytesLimit() (#8794)
    • Support arena allocation of google::protobuf::AnyMetadata (#8758)
    • Fix undefined symbol error around SharedCtor() (#8827)
    • Fix default value of enum(int) in json_util with proto2 (#8835)
    • Better Smaller ByteSizeLong
    • Introduce event filters for inject_field_listener_events
    • Reduce memory usage of DescriptorPool
    • For lazy fields copy serialized form when allowed.
    • Re-introduce the InlinedStringField class
    • v2 access listener
    • Reduce padding in the proto's ExtensionRegistry map.
    • GetExtension performance optimizations
    • Make tracker a static variable rather than call static functions
    • Support extensions in field access listener
    • Annotate MergeFrom for field access listener
    • Fix incomplete types for field access listener
    • Add map_entry/new_map_entry to SpecificField in MessageDifferencer. They record the map items which are different in MessageDifferencer's reporter.
    • Reduce binary size due to fieldless proto messages
    • TextFormat: ParseInfoTree supports getting field end location in addition to start.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump numpy from 1.18.1 to 1.22.0

    Bump numpy from 1.18.1 to 1.22.0

    Bumps numpy from 1.18.1 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v1.6)
Owner
MIT HAN Lab
Accelerating Deep Learning Computing
MIT HAN Lab
PyTorch implementation for OCT-GAN Neural ODE-based Conditional Tabular GANs (WWW 2021)

OCT-GAN: Neural ODE-based Conditional Tabular GANs (OCT-GAN) Code for reproducing the experiments in the paper: Jayoung Kim*, Jinsung Jeon*, Jaehoon L

BigDyL 7 Dec 27, 2022
Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data - Official PyTorch Implementation (CVPR 2022)

Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data (CVPR 2022) Potentials of primitive shapes f

null 31 Sep 27, 2022
DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time

DR-GAN: Automatic Radial Distortion Rectification Using Conditional GAN in Real-Time Introduction This is official implementation for DR-GAN (IEEE TCS

Kang Liao 18 Dec 23, 2022
[CVPR 2021] Pytorch implementation of Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs

Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs In this work, we propose a framework HijackGAN, which enables non-linear latent space travers

Hui-Po Wang 46 Sep 5, 2022
[CVPR 2021] Anycost GANs for Interactive Image Synthesis and Editing

Anycost GAN video | paper | website Anycost GANs for Interactive Image Synthesis and Editing Ji Lin, Richard Zhang, Frieder Ganz, Song Han, Jun-Yan Zh

MIT HAN Lab 726 Dec 28, 2022
[CVPR 2020] Interpreting the Latent Space of GANs for Semantic Face Editing

InterFaceGAN - Interpreting the Latent Space of GANs for Semantic Face Editing Figure: High-quality facial attributes editing results with InterFaceGA

GenForce: May Generative Force Be with You 1.3k Dec 29, 2022
An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects different compression algorithms have.

ImageCompressionSimulation An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects o

James Park 1 Dec 11, 2021
StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

null 3k Jan 8, 2023
Synthesizing and manipulating 2048x1024 images with conditional GANs

pix2pixHD Project | Youtube | Paper Pytorch implementation of our method for high-resolution (e.g. 2048x1024) photorealistic image-to-image translatio

NVIDIA Corporation 6k Dec 27, 2022
Collapse by Conditioning: Training Class-conditional GANs with Limited Data

Collapse by Conditioning: Training Class-conditional GANs with Limited Data Moha

Mohamad Shahbazi 33 Dec 6, 2022
Online Multi-Granularity Distillation for GAN Compression (ICCV2021)

Online Multi-Granularity Distillation for GAN Compression (ICCV2021) This repository contains the pytorch codes and trained models described in the IC

Bytedance Inc. 299 Dec 16, 2022
GAN encoders in PyTorch that could match PGGAN, StyleGAN v1/v2, and BigGAN. Code also integrates the implementation of these GANs.

MTV-TSA: Adaptable GAN Encoders for Image Reconstruction via Multi-type Latent Vectors with Two-scale Attentions. This is the official code release fo

owl 37 Dec 24, 2022
GAN JAX - A toy project to generate images from GANs with JAX

GAN JAX - A toy project to generate images from GANs with JAX This project aims to bring the power of JAX, a Python framework developped by Google and

Valentin Goldité 14 Nov 29, 2022
PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement.

DECOR-GAN PyTorch 1.5 implementation for paper DECOR-GAN: 3D Shape Detailization by Conditional Refinement, Zhiqin Chen, Vladimir G. Kim, Matthew Fish

Zhiqin Chen 72 Dec 31, 2022
Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis

Chunked Autoregressive GAN (CARGAN) Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis [paper] [compan

Descript 150 Dec 6, 2022
UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems

[ICLR 2021] "UMEC: Unified Model and Embedding Compression for Efficient Recommendation Systems" by Jiayi Shen, Haotao Wang*, Shupeng Gui*, Jianchao Tan, Zhangyang Wang, and Ji Liu

VITA 39 Dec 3, 2022
An efficient and easy-to-use deep learning model compression framework

TinyNeuralNetwork 简体中文 TinyNeuralNetwork is an efficient and easy-to-use deep learning model compression framework, which contains features like neura

Alibaba 441 Dec 25, 2022
PyTorch Implementation of DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs

DiffGAN-TTS - PyTorch Implementation PyTorch implementation of DiffGAN-TTS: High

Keon Lee 157 Jan 1, 2023
FuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space OptimizationFuseDream: Training-Free Text-to-Image Generationwith Improved CLIP+GAN Space Optimization

FuseDream This repo contains code for our paper (paper link): FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizat

XCL 191 Dec 31, 2022