Contents
- Semantic Image Synthesis with DAGAN
- Installation
- Dataset Preparation
- Generating Images Using Pretrained Model
- Train and Test New Models
- Evaluation
- Acknowledgments
- Related Projects
- Citation
- Contributions
- Collaborations
Semantic Image Synthesis with DAGAN
Dual Attention GANs for Semantic Image Synthesis
Hao Tang1, Song Bai2, Nicu Sebe13.
1University of Trento, Italy, 2University of Oxford, UK, 3Huawei Research Ireland, Ireland.
In ACM MM 2020.
The repository offers the official implementation of our paper in PyTorch.
In the meantime, check out our related CVPR 2020 paper Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation and Arxiv paper Edge Guided GANs with Semantic Preserving for Semantic Image Synthesis.
Framework
Results of Generated Images
Cityscapes (512×256)
Facades (1024×1024)
ADE20K (256×256)
CelebAMask-HQ (512×512)
Results of Generated Segmenation Maps
License
Copyright (C) 2020 University of Trento, Italy.
All rights reserved. Licensed under the CC BY-NC-SA 4.0 (Attribution-NonCommercial-ShareAlike 4.0 International)
The code is released for academic research use only. For commercial use, please contact [email protected].
Installation
Clone this repo.
git clone https://github.com/Ha0Tang/DAGAN
cd DAGAN/
This code requires PyTorch 1.0 and python 3+. Please install dependencies by
pip install -r requirements.txt
This code also requires the Synchronized-BatchNorm-PyTorch rep.
cd DAGAN_v1/
cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../
To reproduce the results reported in the paper, you would need an NVIDIA DGX1 machine with 8 V100 GPUs.
Dataset Preparation
Please download the datasets on the respective webpages.
- Facades: 55.8M, here.
- DeepFashion: 592.3M, here.
- CelebAMask-HQ: 2.7G, here.
- Cityscapes: 8.4G, here.
- ADE20K: 953.7M, here.
- COCO-Stuff: 21.5G, here.
We also provide the prepared datasets for your convience.
sh datasets/download_dagan_dataset.sh [dataset]
where [dataset] can be one of facades, deepfashion, celeba, cityscapes, ade20k, or coco_stuff.
Generating Images Using Pretrained Model
- Download the pretrained models using the following script,
sh scripts/download_dagan_model.sh GauGAN_DAGAN_[dataset]
where [dataset] can be one of cityscapes, ade, facades, or celeba.
- Change several parameter and then generate images using
test_[dataset].sh. If you are running on CPU mode, append--gpu_ids -1. - The outputs images are stored at
./results/[type]_pretrained/by default. You can view them using the autogenerated HTML file in the directory.
Train and Test New Models
- Prepare dataset.
- Change several parameters and then run
train_[dataset].shfor training. There are many options you can specify. To specify the number of GPUs to utilize, use--gpu_ids. If you want to use the second and third GPUs for example, use--gpu_ids 1,2. - Testing is similar to testing pretrained models. Use
--results_dirto specify the output directory.--how_manywill specify the maximum number of images to generate. By default, it loads the latest checkpoint. It can be changed using--which_epoch.
Evaluation
- FID: mseitzer/pytorch-fid
- FRD: Ha0Tang/GestureGAN
- LPIPS: richzhang/PerceptualSimilarity
- DRN: fyu/drn [model: drn-d-105_ms_cityscapes.pth]
- UperNet: CSAILVision/semantic-segmentation-pytorch [model: baseline-resnet101-upernet]
- DeepLab: kazuto1011/deeplab-pytorch [model: deeplabv2_resnet101_msc-cocostuff164k-100000.pth]
For more details, please refer to this issue.
Acknowledgments
This source code is inspired by both GauGAN/SPADE and LGGAN.
Related Projects
EdgeGAN | LGGAN | SelectionGAN | PanoGAN | Guided-I2I-Translation-Papers
Citation
If you use this code for your research, please consider giving stars
DAGAN
@inproceedings{tang2020dual,
title={Dual Attention GANs for Semantic Image Synthesis},
author={Tang, Hao and Bai, Song and Sebe, Nicu},
booktitle ={ACM MM},
year={2020}
}
EdgeGAN
@article{tang2020edge,
title={Edge Guided GANs with Semantic Preserving for Semantic Image Synthesis},
author={Tang, Hao and Qi, Xiaojuan and Xu, Dan and Torr, Philip HS and Sebe, Nicu},
journal={arXiv preprint arXiv:2003.13898},
year={2020}
}
LGGAN
@inproceedings{tang2019local,
title={Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation},
author={Tang, Hao and Xu, Dan and Yan, Yan and Torr, Philip HS and Sebe, Nicu},
booktitle={CVPR},
year={2020}
}
SelectionGAN
@inproceedings{tang2019multi,
title={Multi-channel attention selection gan with cascaded semantic guidance for cross-view image translation},
author={Tang, Hao and Xu, Dan and Sebe, Nicu and Wang, Yanzhi and Corso, Jason J and Yan, Yan},
booktitle={CVPR},
year={2019}
}
@article{tang2020multi,
title={Multi-channel attention selection gans for guided image-to-image translation},
author={Tang, Hao and Xu, Dan and Yan, Yan and Corso, Jason J and Torr, Philip HS and Sebe, Nicu},
journal={arXiv preprint arXiv:2002.01048},
year={2020}
}
Contributions
If you have any questions/comments/bug reports, feel free to open a github issue or pull a request or e-mail to the author Hao Tang ([email protected]).
Collaborations
I'm always interested in meeting new people and hearing about potential collaborations. If you'd like to work together or get in contact with me, please email [email protected]. Some of our projects are listed here.
Take a few minutes to appreciate what you have and how far you've come.
5 Sep 15, 2022
2 Feb 10, 2022
537 Jan 5, 2023
135 Dec 18, 2022
38.5k Jan 3, 2023
103 Dec 23, 2022
67 Nov 14, 2022
204 Jul 14, 2022
1.8k Dec 30, 2022
247 Jan 5, 2023
214 Dec 7, 2022
829 Jan 7, 2023
490 Dec 15, 2022
230 Dec 7, 2022
7.1k Jan 5, 2023
159 Apr 4, 2022
30 Dec 12, 2022
89 Dec 18, 2022