GANILLA
We provide PyTorch implementation for:
GANILLA: Generative Adversarial Networks for Image to Illustration Translation.
Updates
- (February, 2021) We released code for our recent work on sketch colorization Adversarial Segmentation Loss for Sketch Colorization.
Dataset Stats:
Sample Images:
GANILLA:
GANILLA results on the illustration dataset:
Comparison with other methods:
Style transfer using Miyazaki's anime images:
Ablation Experiments:
Prerequisites
- Linux, macOS or Windows
- Python 2 or 3
- CPU or NVIDIA GPU + CUDA CuDNN
Getting Started
Downloading Datasets
Please refer to datasets.md for details.
Installation
- Clone this repo:
git clone https://github.com/giddyyupp/ganilla.git
cd ganilla
- Install PyTorch 0.4+ and torchvision from http://pytorch.org and other dependencies (e.g., visdom and dominate). You can install all the dependencies by
pip install -r requirements.txt
- For Conda users, we include a script
./scripts/conda_deps.shto install PyTorch and other libraries.
GANILLA train/test
- Download a GANILLA/CycleGAN dataset (e.g. maps):
bash ./datasets/download_cyclegan_dataset.sh maps
- Train a model:
#!./scripts/train_ganilla.sh
python train.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --netG resnet_fpn
- To view training results and loss plots, run
python -m visdom.serverand click the URL http://localhost:8097. To see more intermediate results, check out./checkpoints/maps_cyclegan/web/index.html - Test the model:
#!./scripts/test_cyclegan.sh
python test.py --dataroot ./datasets/maps --name maps_cyclegan --model cycle_gan --netG resnet_fpn
The test results will be saved to a html file here: ./results/maps_cyclegan/latest_test/index.html.
You can find more scripts at scripts directory.
Apply a pre-trained model (GANILLA)
- You can download pretrained models using following link
Put a pretrained model under ./checkpoints/{name}_pretrained/100_net_G.pth.
- To test the model, you also need to download the monet2photo dataset and use trainB images as source:
bash ./datasets/download_cyclegan_dataset.sh monet2photo
- Then generate the results using
python test.py --dataroot datasets/monet2photo/testB --name {name}_pretrained --model test
The option --model test is used for generating results of GANILLA only for one side. python test.py --model cycle_gan will require loading and generating results in both directions, which is sometimes unnecessary. The results will be saved at ./results/. Use --results_dir {directory_path_to_save_result} to specify the results directory.
- If you would like to apply a pre-trained model to a collection of input images (rather than image pairs), please use
--dataset_mode singleand--model testoptions. Here is a script to apply a model to Facade label maps (stored in the directoryfacades/testB).
#!./scripts/test_single.sh
python test.py --dataroot ./datasets/monet2photo/testB/ --name {your_trained_model_name} --model test
You might want to specify --netG to match the generator architecture of the trained model.
Style & Content CNN
We shared style & content CNNs in this repo. It contains train/test procedure as well as pretrained weights for both cnns.
Training/Test Tips
Best practice for training and testing your models.
Frequently Asked Questions
Before you post a new question, please first look at the above Q & A and existing GitHub issues.
Citation
If you use this code for your research, please cite our papers.
@article{hicsonmez2020ganilla,
title={GANILLA: Generative adversarial networks for image to illustration translation},
author={Hicsonmez, Samet and Samet, Nermin and Akbas, Emre and Duygulu, Pinar},
journal={Image and Vision Computing},
pages={103886},
year={2020},
publisher={Elsevier}
}
@inproceedings{Hicsonmez:2017:DDN:3078971.3078982,
author = {Hicsonmez, Samet and Samet, Nermin and Sener, Fadime and Duygulu, Pinar},
title = {DRAW: Deep Networks for Recognizing Styles of Artists Who Illustrate Children's Books},
booktitle = {Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval},
year = {2017}
}
Acknowledgments
Our code is heavily inspired by CycleGAN.
The numerical calculations reported in this work were fully performed at TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).
3.2k Dec 30, 2022
443 Dec 6, 2022
364 Dec 14, 2022
111 Dec 31, 2022
28 Nov 16, 2022
35 Dec 6, 2022
87 Dec 30, 2022
11.3k Dec 30, 2022
34 Jan 19, 2022
307 Jan 3, 2023
217 Jan 4, 2023
123 Dec 27, 2022
173 Dec 21, 2022
489 Jan 7, 2023
153 Dec 14, 2022
97 Dec 23, 2022
71 Nov 18, 2022
165 Dec 28, 2022