A sketch extractor for anime/illustration.

Overview

Anime2Sketch

Anime2Sketch: A sketch extractor for illustration, anime art, manga

By Xiaoyu Xiang

teaser demo

Updates

  • 2021.5.2: Upload more example results of anime video.
  • 2021.4.30: Upload the test scripts. Now our repo is ready to run!
  • 2021.4.11: Upload the pretrained weights, and more test results.
  • 2021.4.8: Create the repo.

Introduction

The repository contains the testing codes and pretrained weights for Anime2Sketch.

Anime2Sketch is a sketch extractor that works well on illustration, anime art, and manga. It is an application based on the paper "Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis".

Prerequisites

Get Started

Installation

Install the required packages: pip install -r requirements.txt

Download Pretrained Weights

Please download the weights from GoogleDrive, and put it into the weights/ folder.

Test

python3 test.py --dataroot /your_input/dir --load_size 512 --output_dir /your_output/dir

The above command includes three arguments:

  • dataroot: your test file or directory
  • load_size: due to the memory limit, we need to resize the input image before processing. By default, we resize it to 512x512.
  • output_dir: path of the output directory

Run our example:

python3 test.py --dataroot test_samples/madoka.jpg --load_size 512 --output_dir results/

Train

This project is a sub-branch of AODA. Please check it for the training instructions.

More Results

Our model works well on illustration arts: madoka demo demo1 Turn handrawn photos to clean linearts: demo2 Simplify freehand sketches: demo3 And more anime results: demo4 demo5

Contact

Xiaoyu Xiang.

You can also leave your questions as issues in the repository. I will be glad to answer them!

License

This project is released under the MIT License.

Citations

@misc{Anime2Sketch,
  author = {Xiaoyu Xiang, Ding Liu, Xiao Yang, Yiheng Zhu, Xiaohui Shen},
  title = {Anime2Sketch: A Sketch Extractor for Anime Arts with Deep Networks},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/Mukosame/Anime2Sketch}}
}

@misc{xiang2021adversarial,
      title={Adversarial Open Domain Adaption for Sketch-to-Photo Synthesis}, 
      author={Xiang, Xiaoyu and Liu, Ding and Yang, Xiao and Zhu, Yiheng and Shen, Xiaohui and Allebach, Jan P},
      year={2021},
      eprint={2104.05703},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
Comments
  • ValueError: Unknown resampling filter

    ValueError: Unknown resampling filter

    When I run test.py, I get a ValueError: Unknown resampling filter (InterpolationMode.BICUBIC) error in data.py. To avoid errors data.py line 96 image_pil = image_pil.resize (output_resize, bic) is image_pil = image_pil.resize (output_resize, Image.BICUBIC) or Shouldn't image_pil = image_pil.resize (output_resize, 3)? This way the error no longer occurs.

    opened by cedro3 4
  • Artifacts in result sketch

    Artifacts in result sketch

    Hi

    Thanks for your great work.

    When I test on my own dataset, sometimes it will show some artifacts in darkness place. Do you have any good suggestions to fix it? artifact

    opened by Tia-Cheung 3
  • line extraction but with colored lines

    line extraction but with colored lines

    Hi, thanks to great works.

    tho found this project has been stalled for a year

    can it do line extraction but with colored lines (mark) , train model on multi-colour lines pairs ?

    In fact, in animation production, called 色トレス, need work that tracing the scanned paper drawing into digital

    the tracing easy to coloring (compare to illustration making), tracing following coloring reference to standardize lines colour, like solid line, shadow, highlight mark

    opened by koke2c95 2
  • 可以使用这个模型用来训练普通图片到草图吗?

    可以使用这个模型用来训练普通图片到草图吗?

    你好,可以使用这个模型用来训练普通图片到草图吗?我使用AODA进行了一些尝试,但是效果并不理想,我猜可能是我哪里做错了。 这是我的训练参数,希望能得到你的帮助,谢谢!

    python train.py --dataroot ./dataset/sketch/ --name scribble_aoda  --model aoda_gan  --gan_mode vanilla --no_dropout --n_classes 9 --direction BtoA --save_epoch_freq 1 --load_size 260 --continue_train --gpu_ids 0 
    
    opened by bakeDong1 2
  • Add docker environment

    Add docker environment

    Summary

    Add Dockerfile and example Makefile. You can run demo just by customizing mount volumes for input/output images directory on Makefile. #3

    Build docker image

    make docker-build

    Run

    Customize mount volumes for input/output directory on Makefile docker run -it --rm --gpus all -v `pwd`:/workspace -v <input images dir>:/input -v <output images dir>:/output anime2sketch Run container make docker-run

    if you run cpu only, you have to fix Dockerfile CMD to CMD [ "python", "test.py", "--dataroot", "/input", "--load_size", "512", "--output_dir", "/output" ] and Makefile docker-run to docker run -it --rm -v `pwd`:/workspace -v `pwd`/images/input:/input -v `pwd`/images/output:/output anime2sketch

    opened by kitoriaaa 2
  • ValueError: Unknown resampling filter error

    ValueError: Unknown resampling filter error

    I'm using Windows 10 and python 3.9

    PS C:\Users\Tamal\Desktop\Anime2Sketch> python3 test.py --dataroot image --load_size 512 --output_dir results Traceback (most recent call last): File "C:\Users\Tamal\Desktop\Anime2Sketch\test.py", line 46, in save_image(aus_img, aus_path, aus_resize) File "C:\Users\Tamal\Desktop\Anime2Sketch\data.py", line 96, in save_image image_pil = image_pil.resize(output_resize, bic) File "C:\Users\Tamal\AppData\Local\Programs\Python\Python39\lib\site-packages\PIL\Image.py", line 1957, in resize raise ValueError( ValueError: Unknown resampling filter (InterpolationMode.BICUBIC). Use Image.NEAREST (0), Image.LANCZOS (1), Image.BILINEAR (2), Image.BICUBIC (3), Image.BOX (4) or Image.HAMMING (5) PS C:\Users\Tamal\Desktop\Anime2Sketch>

    opened by tamal777 2
  • how to train for my own dataset

    how to train for my own dataset

    Hi, thanks for your sharing , I want to train my own dataset using this network

    I see in readme for training in AODA, but AODA repo also has nothing about how to train 。。。

    opened by Johnson-yue 2
  • Error when using Gradio

    Error when using Gradio

    Hello, just like the title said. I can't perform Anime2Sketch using the Gradio demo website and I got an error.

    image

    And I can't use the demo image from the website.

    image

    I don't know it is a bug or just my internet because I already try to delete the cookies and I still have that trouble.

    Thank you

    opened by shandytp 2
  • Dataset for training

    Dataset for training

    Hello, I love your research and output is so great

    I'm just wondering how mach size of dataset are used for pretrained model of this Anime2sketch, through the original code https://github.com/Mukosame/AODA

    waiting for reply

    Thank you

    opened by Chanuku 1
  • User Warning

    User Warning

    I am getting the following error while trying the test code.

    UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum. warnings.warn(

    opened by gmehta1996 1
  • How can we make it work on normal images?

    How can we make it work on normal images?

    Hi, I would like to convert normal face images to sketch. Would a model trained on ffhq dataset perform well with normal images? Have you tried something like this before?

    opened by Nerdyvedi 1
  • Fixed error when applying a resampling filter.

    Fixed error when applying a resampling filter.

    This fix prevent the following error happening when resizing input image(s) before generating sketches: ValueError: Unknown resampling filter (InterpolationMode.BICUBIC). Use Image.Resampling.NEAREST (0), Image.Resampling.LANCZOS (1), Image.Resampling.BILINEAR (2), Image.Resampling.BICUBIC (3), Image.Resampling.BOX (4) or Image.Resampling.HAMMING (5)

    opened by virtualramblas 0
  • ValueError: Unknown resampling filter (InterpolationMode.BICUBIC).

    ValueError: Unknown resampling filter (InterpolationMode.BICUBIC).

    ValueError: Unknown resampling filter (InterpolationMode.BICUBIC). Use Image.Resampling.NEAREST (0), Image.Resampling.LANCZOS (1), Image.Resampling.BILINEAR (2), Image.Resampling.BICUBIC (3), Image.Resampling.BOX (4) or Image.Resampling.HAMMING (5)

    opened by geng-lee 1
  • ValueError: Expected more than 1 spatial element when training, got input size torch.Size([1, 512, 1, 1])

    ValueError: Expected more than 1 spatial element when training, got input size torch.Size([1, 512, 1, 1])

    raise ValueError("Expected more than 1 spatial element when training, got input size {}".format(size)) ValueError: Expected more than 1 spatial element when training, got input size torch.Size([1, 512, 1, 1])

    analytics-python 1.4.0 backoff 1.10.0 bcrypt 3.2.0 certifi 2021.10.8 cffi 1.15.0 charset-normalizer 2.0.8 click 8.0.3 colorama 0.4.4 cryptography 36.0.0 cycler 0.11.0 ffmpy 0.3.0 Flask 2.0.2 Flask-CacheBuster 1.0.0 Flask-Cors 3.0.10 Flask-Login 0.5.0 fonttools 4.28.2 gradio 2.4.6 idna 3.3 itsdangerous 2.0.1 Jinja2 3.0.3 kiwisolver 1.3.2 markdown2 2.4.1 MarkupSafe 2.0.1 matplotlib 3.5.0 monotonic 1.6 numpy 1.21.4 packaging 21.3 pandas 1.3.4 paramiko 2.8.1 pep517 0.12.0 Pillow 8.4.0 pip 21.3.1 pycparser 2.21 pycryptodome 3.11.0 pydub 0.25.1 PyNaCl 1.4.0 pyparsing 3.0.6 python-dateutil 2.8.2 pytz 2021.3 requests 2.26.0 setuptools 41.2.0 setuptools-scm 6.3.2 six 1.16.0 tomli 1.2.2 torch 1.10.0 torchtext 0.11.0 torchvision 0.11.1 tqdm 4.62.3 typing_extensions 4.0.0 urllib3 1.26.7 Werkzeug 2.0.2

    opened by xcfstudio 0
  • Why dont you add a simple bat file for windows to go through all files in folder ?

    Why dont you add a simple bat file for windows to go through all files in folder ?

    heres what works for me and converts all files from in folder to out folder with 2048 tex quality :

    @echo off for %%F in (in*.*) do test.py --dataroot in --load_size 2048 --output_dir out/

    opened by 2blackbar 0
Owner
Xiaoyu Xiang
Ph.D. candidate at Purdue University.
Xiaoyu Xiang
Metadata-Extractor - Metadata Extractor Script can be used to read in exif metadata

Metadata Extractor The exifextract script can be used to read in exif metadata f

null 1 Feb 16, 2022
A highly efficient, fast, powerful and light-weight anime downloader and streamer for your favorite anime.

AnimDL - Download & Stream Your Favorite Anime AnimDL is an incredibly powerful tool for downloading and streaming anime. Core features Abuses the dev

KR 759 Jan 8, 2023
Little tool in python to watch anime from the terminal (the better way to watch anime)

ani-cli Script working again :), thanks to the fork by Dink4n for the alternative approach to by pass the captcha on gogoanime A cli to browse and wat

Harshith 4.5k Dec 31, 2022
A code repository associated with the paper A Benchmark for Rough Sketch Cleanup by Chuan Yan, David Vanderhaeghe, and Yotam Gingold from SIGGRAPH Asia 2020.

A Benchmark for Rough Sketch Cleanup This is the code repository associated with the paper A Benchmark for Rough Sketch Cleanup by Chuan Yan, David Va

null 33 Dec 18, 2022
[CVPR 21] Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting, IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021.

Vectorization and Rasterization: Self-Supervised Learning for Sketch and Handwriting, CVPR 2021. Ayan Kumar Bhunia, Pinaki nath Chowdhury, Yongxin Yan

Ayan Kumar Bhunia 44 Dec 12, 2022
Compositional Sketch Search

Compositional Sketch Search Official repository for ICIP 2021 Paper: Compositional Sketch Search Requirements Install and activate conda environment c

Alexander Black 8 Sep 6, 2021
Code for the paper: Sketch Your Own GAN

Sketch Your Own GAN Project | Paper | Youtube Our method takes in one or a few hand-drawn sketches and customizes an off-the-shelf GAN to match the in

null 677 Dec 28, 2022
A pytorch implementation of Pytorch-Sketch-RNN

Pytorch-Sketch-RNN A pytorch implementation of https://arxiv.org/abs/1704.03477 In order to draw other things than cats, you will find more drawing da

Alexis David Jacq 172 Dec 12, 2022
PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence) and pre-trained model on ImageNet dataset

Reference-Based-Sketch-Image-Colorization-ImageNet This is a PyTorch implementation of CVPR 2020 paper (Reference-Based Sketch Image Colorization usin

Yuzhi ZHAO 11 Jul 28, 2022
GANSketchingJittor - Implementation of Sketch Your Own GAN in Jittor

GANSketching in Jittor Implementation of (Sketch Your Own GAN) in Jittor(计图). Or

Bernard Tan 10 Jul 2, 2022
Open CV - Convert a picture to look like a cartoon sketch in python

Use the video https://www.youtube.com/watch?v=k7cVPGpnels for initial learning.

Sammith S Bharadwaj 3 Jan 29, 2022
An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects different compression algorithms have.

ImageCompressionSimulation An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects o

James Park 1 Dec 11, 2021
Table-Extractor 表格抽取

(t)able-(ex)tractor 本项目旨在实现pdf表格抽取。 Models 版面分析模块(Yolo) 表格结构抽取(ResNet + Transformer) 文字识别模块(CRNN + CTC Loss) Acknowledgements TableMaster attention-i

null 2 Jan 15, 2022
Snscrape-jsonl-urls-extractor - Extracts urls from jsonl produced by snscrape

snscrape-jsonl-urls-extractor extracts urls from jsonl produced by snscrape Usag

null 1 Feb 26, 2022
For holding anime-related object classification and detection models

Animesion An end-to-end framework for anime-related object classification, detection, segmentation, and other models. Update: 01/22/2020. Due to time-

Edwin Arkel Rios 72 Nov 30, 2022
[Open Source]. The improved version of AnimeGAN. Landscape photos/videos to anime

[Open Source]. The improved version of AnimeGAN. Landscape photos/videos to anime

CC 4.4k Dec 27, 2022
StyleGAN2 Webtoon / Anime Style Toonify

StyleGAN2 Webtoon / Anime Style Toonify Korea Webtoon or Japanese Anime Character Stylegan2 base high Quality 1024x1024 / 512x512 Generate and Transfe

null 121 Dec 21, 2022
Generating Anime Images by Implementing Deep Convolutional Generative Adversarial Networks paper

AnimeGAN - Deep Convolutional Generative Adverserial Network PyTorch implementation of DCGAN introduced in the paper: Unsupervised Representation Lear

Rohit Kukreja 23 Jul 21, 2022
A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing.

AnimeGAN A simple PyTorch Implementation of Generative Adversarial Networks, focusing on anime face drawing. Randomly Generated Images The images are

Jie Lei 雷杰 1.2k Jan 3, 2023