The pytorch implementation of DG-Font: Deformable Generative Networks for Unsupervised Font Generation

Related tags

Deep Learning DG-Font
Overview

DG-Font: Deformable Generative Networks for Unsupervised Font Generation

The source code for 'DG-Font: Deformable Generative Networks for Unsupervised Font Generation', by Yangchen Xie, Xinyuan Chen, Li sun and Yue lu. The paper was accepted by CVPR2021.

Arvix version

Gallery

image image

Dependencies

Libarary

pytorch (>=1.0)
tqdm  
numpy
opencv-python  
scipy  
sklearn
matplotlib  
pillow  
tensorboardX

DCN

please refer to https://github.com/chengdazhi/Deformable-Convolution-V2-PyTorch/tree/pytorch_1.0.0 to install the dependencies of deformable convolution.

Dataset

方正字库 provides free font download for non-commercial users.

Example directory hierarchy

Project
|--- DG-Font
|          |--- font2img.py    
|          |--- main.py
|          |--- train
|                 |--- train.py
|
|--- data
       |--- font1
       |--- font2
             |--- 0000.png
             |--- 0001.png
             |--- ...
       |--- ...

How to run

prepare dataset

python font2img.py --ttf_path ttf_folder --chara character.txt --save_path save_folder --img_size 80 --chara_size CHARACTERSIZE

train

python main.py --gpu GPU_ID --img_size 80 --data_path /path/to --output_k CLASS_NUM --batch_size BATCHSIZE --val_num TEST_IMGS_NUM_FOR_EACH_CLASS

test

python main.py --gpu GPU_ID --img_size 80 --data_path /path/to --output_k CLASS_NUM --batch_size BATCHSIZE --validation --load_model $DIR_TO_LOAD

Acknowledgements

We would like to thank Johnson yue and 上海驿创信息技术有限公司 for their advices in code. Our code is based on TUNIT.

Bibtex

@inproceedings{DG-Font,
    title={DG-Font: Deformable Generative Networks for Unsupervised Font Generation},
    author={Yangchen Xie, Xinyuan Chen, Li sun, Yue lu},
    booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
    year={2021}
}
Comments
  • error occur in validation

    error occur in validation

    Traceback (most recent call last): File "main.py", line 400, in main() File "main.py", line 170, in main main_worker(args.gpu, ngpus_per_node, args) File "main.py", line 250, in main_worker validationFunc(val_loader, networks, epoch, args, {'logger': logger}) File "/content/drive/My Drive/DG-Font-main/validation/validation.py", line 82, in validateUN x_res_ema_tmp,_ = G_EMA.decode(c_src, s_ref, skip1, skip2) File "/content/drive/My Drive/DG-Font-main/models/generator.py", line 51, in decode out = self.decoder(cnt, skip1, skip2) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/content/drive/My Drive/DG-Font-main/models/generator.py", line 87, in forward output = self.modeli File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/content/drive/My Drive/DG-Font-main/models/blocks.py", line 15, in forward return self.model(x) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py", line 139, in forward input = module(input) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/content/drive/My Drive/DG-Font-main/models/blocks.py", line 32, in forward residual = self.model(x) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/container.py", line 139, in forward input = module(input) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/content/drive/My Drive/DG-Font-main/models/blocks.py", line 158, in forward x = self.norm(x) File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 1051, in _call_impl return forward_call(*input, **kwargs) File "/content/drive/My Drive/DG-Font-main/models/blocks.py", line 208, in forward True, self.momentum, self.eps) File "/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py", line 2282, in batch_norm input, weight, bias, running_mean, running_var, training, momentum, eps, torch.backends.cudnn.enabled RuntimeError: weight should contain 512 elements not 2560

    opened by zhangfengyo 8
  • Questions about the training process

    Questions about the training process

    Questions about the training process

    Hello, during the training process, when epoch[2 / 250] 100% is reached, the training did not continue. In the result folder, character images are continuously generated, which takes a lot of time. Duration of this process: 9.2~17:00-now. May I ask if this is normal? In addition, is it convenient to inform you of your GPU information and the total training time?

    log

    Epoch: [2/250] [900/1000] MODE[GAN_EMA] Avg Loss: D[2.04] G[0.03] 100%|████████████████████████████████████████████████████████████████████████████████▉| 999/1000 [34:33<00:02, 2.08s/it]Epoch: [2/250] [1000/1000] MODE[GAN_EMA] Avg Loss: D[2.03] G[0.04] 100%|████████████████████████████████████████████████████████████████████████████████| 1000/1000 [34:35<00:00, 2.08s/it]

    Now, there is no output in the terminal. In the result file, images are continuously generated.

    opened by Fyzjym 6
  • How to batch infer character image from a txt file?

    How to batch infer character image from a txt file?

    Thanks for your wonderful work! I have finished the trainning, but I do not know how to infer lots of character via a txt file by a special font type ? I did these works: 1\ trained the repo and finished it. python main.py --gpu 0 --img_size 80 --data_path ./data --output_k 4 --batch_size 8 --val_num190 2\ use font2img.py to change the character into image , the dir is data, and the number of character is 52.

    3\ python main.py --gpu 0 --img_size 80 --data_path ./data --output_k 4 --batch_size 8 --validation --load_model GAN_20211130-124006.

    4\ finnally get 32 images with only 10 character in them. but not 52 .

    question: 1\ how to get all of these character in images. 2\ actually I just want the special style transfer, example font2 to font 4 , how can I do this work?

    Could you help to give some advice ? Many thanks for your kindly help!

    opened by ApolloZhangCN 2
  • Request the character.txt & your CHARACTERSIZE setting

    Request the character.txt & your CHARACTERSIZE setting

    Hello, will you share the character.txt, the setting CHARACTERSIZE and the exact font name that was used to generate your train & test data? It will be much helpful for reproducing the results reported in the paper.

    Thanks in advance.

    opened by songquanpeng 2
  • Question about fig.7 in paper

    Question about fig.7 in paper

    Hi,sir. I have the same question with #6. But to be honest, I don't understand how this picture is implemented... I'm very sorry, can you elaborate on the implementation details... or are there any examples. Thanks for reading

    opened by Fyzjym 2
  • how to generate a font file use image?

    how to generate a font file use image?

    @ecnuycxie Thanks for sharing your nice work! I tried to use the method you mentioned to generate a font image of a certain style. And now I got 3500 font images of single chinese character, but how to make a .ttf font file use this images? Because I cann't use 3500 images to do design, .ttf or .ttc file will be more easy to use.

    opened by dagongji10 2
  • RuntimeError: weight should contain 1536 elements not 2560

    RuntimeError: weight should contain 1536 elements not 2560

    When I tried to test the model, I got the following error:

    Traceback (most recent call last): File "main.py", line 402, in main() File "main.py", line 170, in main main_worker(args.gpu, ngpus_per_node, args) File "main.py", line 221, in main_worker validationFunc(val_loader, networks, 999, args, {'logger': logger}) File "/home/ai/DG-Font-main/validation/validation.py", line 80, in validateUN x_res_ema_tmp,_ = G_EMA.decode(c_src, s_ref, skip1, skip2) File "/home/ai/DG-Font-main/models/generator.py", line 51, in decode out = self.decoder(cnt, skip1, skip2) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/ai/DG-Font-main/models/generator.py", line 87, in forward output = self.modeli File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/ai/DG-Font-main/models/blocks.py", line 15, in forward return self.model(x) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/container.py", line 100, in forward input = module(input) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/ai/DG-Font-main/models/blocks.py", line 32, in forward residual = self.model(x) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/container.py", line 100, in forward input = module(input) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/ai/DG-Font-main/models/blocks.py", line 162, in forward x = self.norm(x) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in call result = self.forward(*input, **kwargs) File "/home/ai/DG-Font-main/models/blocks.py", line 212, in forward True, self.momentum, self.eps) File "/home/anaconda3/envs/zi2zi-pytorch/lib/python3.7/site-packages/torch/nn/functional.py", line 1923, in batch_norm training, momentum, eps, torch.backends.cudnn.enabled RuntimeError: weight should contain 1536 elements not 2560

    my validation cmd is: python main.py --gpu 0 --img_size 80 --data_path ./hk_data/save --output_k 2 --batch_size 32 --validation --load_model ./logs/GAN_20210908-163125 --val_num 6

    Thank you!

    opened by blackpill 2
  • Epochs or styles(fonts)? Which factor is more important for the model performance?

    Epochs or styles(fonts)? Which factor is more important for the model performance?

    If we wanna get better performance for unseen fonts generation, what we should do when training the model?

    Is it helpful when we increase the number of training styles?

    How many epochs we should train at least?

    opened by blackpill 1
  • How many characters are required to generate unseen font characters?

    How many characters are required to generate unseen font characters?

    Hello, thank you for your wonderful work.

    Could you tell me how many characters as style-image are required to generate characters with an unseen font? According to the issue, 20 seems not enough.

    opened by yukistavailable 0
  • Any trained model?

    Any trained model?

    Hi @ecnuycxie Thank you for your work. Could you share a checkpoint of the trained model? I couldn't reproduce the scores on your paper, so I want to see how well it worked for you.

    opened by qqpann 0
  • How to installation DCN

    How to installation DCN

    The DCN link given by github can't be installed. I installed other versions of DCN, but the other versions are missing the functions modulated_deform_conv_forward and DCN.modulated_deform_conv_backward that you call. Can you provide me with the installation method of the version of DCN you are using, or provide these two functions?

    opened by zj916716524 4
  • Inquery about the details of dataset (name of fonts and num of characters)

    Inquery about the details of dataset (name of fonts and num of characters)

    Hello! Thanks for your excellent work and your effort on sharing this code 👍 . Here I have some questions when trying to re-implement it:

    1. According to the paper, 410 fonts are used for experiments. So what is the exact name of the used fonts?
    2. The 410 fonts are randomly splitted into two parts (i.e. 400 and 10). So what is the exact font name of these two parts?
    3. As is proposed by another issue, the 990 commonly-used fonts can be retrieved at https://www.aige.co.uk/resources.asp?id=1. However, I found that there are 1000 characters in total. So which 10 characters are not included?

    Again, thanks for reading this issue. I am looking forward to your reply :D

    opened by JingyeChen 0
  • 关于论文图7

    关于论文图7

    作者你好， 论文的图7非常清晰的说明FDSC的作用，我想知道图是如何实现的。

    1. 论文中提到将可变形卷积的卷积核设置为1*1，这是否意味着需要重新训练够才能可视化offsets呢？
    2. 通过将可变形卷积的卷积核设置为1*1后，我得到了一个（1，2，80，80）的特征图，我应该如何将这个特征图可视化呢，实现这个功能的代码可以提供吗？ 非常感谢您的工作，期待您的回复！
    opened by Deepseabro 1
Owner
null
GitHub
Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

Deformable Attention Implementation of Deformable Attention from this paper in Pytorch, which appears to be an improvement to what was proposed in DET

Phil Wang 128 Dec 24, 2022
A multi-scale unsupervised learning for deformable image registration

A multi-scale unsupervised learning for deformable image registration Shuwei Shao, Zhongcai Pei, Weihai Chen, Wentao Zhu, Xingming Wu and Baochang Zha

ShuweiShao 2 Apr 13, 2022
Minimal PyTorch implementation of Generative Latent Optimization from the paper "Optimizing the Latent Space of Generative Networks"

Minimal PyTorch implementation of Generative Latent Optimization This is a reimplementation of the paper Piotr Bojanowski, Armand Joulin, David Lopez-

Thomas Neumann 117 Nov 27, 2022
PyTorch implementation of Deformable Convolution

Deformable Convolutional Networks in PyTorch This repo is an implementation of Deformable Convolution. Ported from author's MXNet implementation. Buil

null 411 Dec 16, 2022
PyTorch implementation of Deformable Convolution

PyTorch implementation of Deformable Convolution !!!Warning: There is some issues in this implementation and this repo is not maintained any more, ple

Wei Ouyang 893 Dec 18, 2022
StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

StudioGAN is a Pytorch library providing implementations of representative Generative Adversarial Networks (GANs) for conditional/unconditional image generation.

null 3k Jan 8, 2023
pytorch implementation of "Contrastive Multiview Coding", "Momentum Contrast for Unsupervised Visual Representation Learning", and "Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination"

Unofficial implementation: MoCo: Momentum Contrast for Unsupervised Visual Representation Learning (Paper) InsDis: Unsupervised Feature Learning via N

Zhiqiang Shen 16 Nov 4, 2020
Official implementation of NPMs: Neural Parametric Models for 3D Deformable Shapes - ICCV 2021

NPMs: Neural Parametric Models Project Page | Paper | ArXiv | Video NPMs: Neural Parametric Models for 3D Deformable Shapes Pablo Palafox, Aljaz Bozic

PabloPalafox 109 Nov 22, 2022
This is the code for Deformable Neural Radiance Fields, a.k.a. Nerfies.

Deformable Neural Radiance Fields This is the code for Deformable Neural Radiance Fields, a.k.a. Nerfies. Project Page Paper Video This codebase conta

Google 1k Jan 9, 2023
[CVPRW 2021] Code for Region-Adaptive Deformable Network for Image Quality Assessment

RADN [CVPRW 2021] Code for Region-Adaptive Deformable Network for Image Quality Assessment [Paper on arXiv] Overview Update [2021/5/7] add codes for W

IIGROUP 53 Dec 28, 2022
Deformable DETR is an efficient and fast-converging end-to-end object detector.

Deformable DETR: Deformable Transformers for End-to-End Object Detection.

null 2k Jan 5, 2023
DPT: Deformable Patch-based Transformer for Visual Recognition (ACM MM2021)

DPT This repo is the official implementation of DPT: Deformable Patch-based Transformer for Visual Recognition (ACM MM2021). We provide code and model

CASIA-IVA-Lab 111 Dec 21, 2022
Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

ARAPReg Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators.. Installation The cod

Bo Sun 132 Nov 28, 2022
Some code of the implements of Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network

3D-GMPDCNN Geological Modeling Using 3D Pixel-Adaptive and Deformable Convolutional Neural Network PyTorch implementation of "Geological Modeling Usin

null 5 Nov 21, 2022
MoCoPnet - Deformable 3D Convolution for Video Super-Resolution

Deformable 3D Convolution for Video Super-Resolution Pytorch implementation of l

Xinyi Ying 28 Dec 15, 2022
3D2Unet: 3D Deformable Unet for Low-Light Video Enhancement (PRCV2021)

3DDUNET This is the code for 3D2Unet: 3D Deformable Unet for Low-Light Video Enhancement (PRCV2021) Conference Paper Link Dataset We use SMOID dataset

null 1 Jan 7, 2022
Unofficial implementation of Alias-Free Generative Adversarial Networks. (https://arxiv.org/abs/2106.12423) in PyTorch

alias-free-gan-pytorch Unofficial implementation of Alias-Free Generative Adversarial Networks. (https://arxiv.org/abs/2106.12423) This implementation

Kim Seonghyeon 502 Jan 3, 2023
Pytorch implementation for reproducing StackGAN_v2 results in the paper StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks

StackGAN-v2 StackGAN-v1: Tensorflow implementation StackGAN-v1: Pytorch implementation Inception score evaluation Pytorch implementation for reproduci

Han Zhang 809 Dec 16, 2022
PyTorch implementation of "Learning to Discover Cross-Domain Relations with Generative Adversarial Networks"

DiscoGAN in PyTorch PyTorch implementation of Learning to Discover Cross-Domain Relations with Generative Adversarial Networks. * All samples in READM

Taehoon Kim 1k Jan 4, 2023