In train_nr.py#L35 you pass an argument named n_blocks, but neural_rasterizer.py shows that an argument with this name does not exist in NeuralRasterizer.__init__'s arguments. Should the value be passed to n_upsampling?

It seems that the font_ttfs.zip includes many fonts and only a subset of it is used for training and testing. So could you please provide the font list you used?

Could you provide the extra training data for image superresolution and neural rasterizer? The missing data seem to includes./data/mean.npz, ./data/stdev.npz and ./data/glyphss_dataset. Thanks!

I used ‘data/utils/convert_ttf_to_sfd_mp.py’ and add '0123456789' after charset, so the finally charset is "a-zA-Z0-9".

when convert "a-zA-Z" it is success, but it failed when char is "0"
TypeError:No such glyph
the error code is line 61 char_description.write(str(new_font_for_char[char].width) + "\n")

I check the font and it has "0" glyph in ttf file

Can the generated svg sequences from the network be used to generate a font file (e.g., .ttf file) directly for use? Will the font size and other factors affect the visual effect in actual use?

I've customized my own dataset and I'm getting errors loading mean.npz and stdev.npz when running main.py. I found out that the type of the loaded data was numpy.lib.npyio.NpzFile. The mean and stdev files downloaded from Google drive seem to be actually .npy format, so the problem does not occur. write_data_to_pkl.py saves them by np.savez, so for example it seems that the load side can support both formats if you do as follows:

@@ -89,9 +89,14 @@ def train_main_model(opts):
 
     if opts.tboard:
         writer = SummaryWriter(log_dir)
-    
-    mean = np.load('./data/mean.npz')
-    std = np.load('./data/stdev.npz')
+
+    def extract_npz_if_needed(data):
+        if not isinstance(data, np.ndarray):
+            data = data[data.files[0]]
+        return data
+
+    mean = extract_npz_if_needed(np.load('./data/mean.npz'))
+    std = extract_npz_if_needed(np.load('./data/stdev.npz'))
     mean = torch.from_numpy(mean).to(device).to(torch.float32)
     std = torch.from_numpy(std).to(device).to(torch.float32)
     network_modules= [img_encoder, img_decoder, modality_fusion, vggptlossfunc, svg_encoder, svg_decoder, mdn_top_layer, neural_rasterizer]

I am using the dataset for clustering and could not understand the structure of cur_glyph['class'] in dataloader.py. I want to extract each glyph and its class labels. Can some guide me?

Hi, I'm encountering the problem in the title.

For example, if mp.cpu_count()=8 and you have 8 fonts under font_ttfs/test, one font will not be processed. In this case, process_nums=6 and font_num_per_process=1.

Processes = [mp.Process(target=process, args=(pid, font_num_per_process)) for pid in range(process_nums + 1)].

will launch 7 processes, but they will each process one font, so only 7 fonts will be processed in total.

It's not a pretty solution, but it seems to work by having the process with the largest pid handle all the remaining fonts, as shown below:

--- a/data_utils/convert_ttf_to_sfd_mp.py
+++ b/data_utils/convert_ttf_to_sfd_mp.py
@@ -26,8 +26,8 @@ def convert_mp(opts):
     else:
         font_num_per_process = font_num // process_nums
 
-    def process(process_id, font_num_p_process):
-        for i in range(process_id * font_num_p_process, (process_id + 1) * font_num_p_process):
+    def process(process_id, font_num_p_process, last=False):
+        for i in range(process_id * font_num_p_process, (process_id + 1) * font_num_p_process if not last else font_num):
             if i >= font_num:
                 break
 
@@ -72,7 +72,7 @@ def convert_mp(opts):
 
             cur_font.close()
 
-    processes = [mp.Process(target=process, args=(pid, font_num_per_process)) for pid in range(process_nums + 1)]
+    processes = [mp.Process(target=process, args=(pid, font_num_per_process, pid==process_nums)) for pid in range(process_nums + 1)]
 
     for p in processes:
         p.start()

Thanks for your impressive work！ however, I have a question about the Neural Differentiable Rasterizer, when you train it, why you randomly select a class of target vector glyph rather than train all of them at the same time? I will apperciate it a lot if you can help me! Thanks a lot!

For anyone who installed diffvg before 2021.10.19: I forgot to mention that we added a new function: save_svg_paths_only in diffvg/pydiffvg/save_svg.py. You need first replace the original diffvg/pydiffvg/save_svg.py with this and then reinstall.

Just a note for anyone attempting to train the model on OS X and coming across this error message - due to an issue in torch (see e.g. https://github.com/pyg-team/pytorch_geometric/issues/366) you will need to replace num_workers=batch_size with num_workers=0 in here: https://github.com/yizhiwang96/deepvecfont/blob/e52166bfd5ffbcd4bd705e9a53fc578e1f956673/dataloader.py#L49 (Yes, this will seriously kill performance.)

This is such an amazing project！ How did you pre-train the NDR model? What are the labels for the training set and the test set?Can the NDR you provide be used to generate vector English characters for each style? Do I need to retrain NDR for each style English Fonts? How to train my own NDR for icons or Chinese characteres？What are the labels for the training set and the test set? I hope to get your reply, thanks!!!

I downloaded link "The Neural Rasterizer :image size 128x128"，"The Image Super-resolution model image size 128x128 -> 256x256"，"The Main model image size 128x128"，and put them in the corresponding folder，To train our main model, run python main.py --mode train --experiment_name dvf --model_name main_model , The following error appears

(dvf) miao@miao:~/data/file/bwt/deepvecfont-master$ python main.py --mode train --experiment_name dvf --model_name main_model Training on experiment dvf_main_model... Loading data/vecfont_dataset_pkls/train/train_all.pkl pickle file ... Finished loading pkls Loading data/vecfont_dataset_pkls/test/test_all.pkl pickle file ... Finished loading pkls Traceback (most recent call last): File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 351, in main() File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 345, in main train(opts) File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 320, in train train_main_model(opts) File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 99, in train_main_model for idx, data in enumerate(train_loader): File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data return self._process_data(data) File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data data.reraise() File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/_utils.py", line 425, in reraise raise self.exc_type(msg) RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/miao/data/file/bwt/deepvecfont-master/dataloader.py", line 52, in getitem item['rendered'] = torch.FloatTensor(cur_glyph['rendered']).view(self.char_num, self.img_size, self.img_size) / 255. RuntimeError: shape '[52, 128, 128]' is invalid for input of size 212992

The following error occurs when I change image_size default=128 in line 13 of options.py to 64

(dvf) miao@miao:~/data/file/bwt/deepvecfont-master$ python main.py --mode train --experiment_name dvf --model_name main_model Training on experiment dvf_main_model... Loading data/vecfont_dataset_pkls/train/train_all.pkl pickle file ... Finished loading pkls Loading data/vecfont_dataset_pkls/test/test_all.pkl pickle file ... Finished loading pkls Traceback (most recent call last): File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 351, in main() File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 345, in main train(opts) File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 320, in train train_main_model(opts) File "/home/miao/data/file/bwt/deepvecfont-master/main.py", line 64, in train_main_model neural_rasterizer.load_state_dict(torch.load(neural_rasterizer_fpath)) File "/home/miao/data/soft/python/anaconda/install/envs/dvf/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1406, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for NeuralRasterizer: Unexpected key(s) in state_dict: "decode.21.weight", "decode.21.bias", "decode.19.weight", "decode.19.bias". size mismatch for decode.0.weight: copying a param with shape torch.Size([1024, 1024, 3, 3]) from checkpoint, the shape in current model is torch.Size([1024, 512, 3, 3]). size mismatch for decode.0.bias: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([512]). size mismatch for decode.1.weight: copying a param with shape torch.Size([1024, 2, 2]) from checkpoint, the shape in current model is torch.Size([512, 2, 2]). size mismatch for decode.1.bias: copying a param with shape torch.Size([1024, 2, 2]) from checkpoint, the shape in current model is torch.Size([512, 2, 2]). size mismatch for decode.3.weight: copying a param with shape torch.Size([1024, 512, 3, 3]) from checkpoint, the shape in current model is torch.Size([512, 256, 3, 3]). size mismatch for decode.3.bias: copying a param with shape torch.Size([512]) from checkpoint, the shape in current model is torch.Size([256]). size mismatch for decode.4.weight: copying a param with shape torch.Size([512, 4, 4]) from checkpoint, the shape in current model is torch.Size([256, 4, 4]). size mismatch for decode.4.bias: copying a param with shape torch.Size([512, 4, 4]) from checkpoint, the shape in current model is torch.Size([256, 4, 4]). size mismatch for decode.6.weight: copying a param with shape torch.Size([512, 256, 5, 5]) from checkpoint, the shape in current model is torch.Size([256, 128, 5, 5]). size mismatch for decode.6.bias: copying a param with shape torch.Size([256]) from checkpoint, the shape in current model is torch.Size([128]). size mismatch for decode.7.weight: copying a param with shape torch.Size([256, 8, 8]) from checkpoint, the shape in current model is torch.Size([128, 8, 8]). size mismatch for decode.7.bias: copying a param with shape torch.Size([256, 8, 8]) from checkpoint, the shape in current model is torch.Size([128, 8, 8]). size mismatch for decode.9.weight: copying a param with shape torch.Size([256, 128, 5, 5]) from checkpoint, the shape in current model is torch.Size([128, 64, 5, 5]). size mismatch for decode.9.bias: copying a param with shape torch.Size([128]) from checkpoint, the shape in current model is torch.Size([64]). size mismatch for decode.10.weight: copying a param with shape torch.Size([128, 16, 16]) from checkpoint, the shape in current model is torch.Size([64, 16, 16]). size mismatch for decode.10.bias: copying a param with shape torch.Size([128, 16, 16]) from checkpoint, the shape in current model is torch.Size([64, 16, 16]). size mismatch for decode.12.weight: copying a param with shape torch.Size([128, 64, 5, 5]) from checkpoint, the shape in current model is torch.Size([64, 32, 5, 5]). size mismatch for decode.12.bias: copying a param with shape torch.Size([64]) from checkpoint, the shape in current model is torch.Size([32]). size mismatch for decode.13.weight: copying a param with shape torch.Size([64, 32, 32]) from checkpoint, the shape in current model is torch.Size([32, 32, 32]). size mismatch for decode.13.bias: copying a param with shape torch.Size([64, 32, 32]) from checkpoint, the shape in current model is torch.Size([32, 32, 32]). size mismatch for decode.15.weight: copying a param with shape torch.Size([64, 32, 5, 5]) from checkpoint, the shape in current model is torch.Size([32, 16, 5, 5]). size mismatch for decode.15.bias: copying a param with shape torch.Size([32]) from checkpoint, the shape in current model is torch.Size([16]). size mismatch for decode.16.weight: copying a param with shape torch.Size([32, 64, 64]) from checkpoint, the shape in current model is torch.Size([16, 64, 64]). size mismatch for decode.16.bias: copying a param with shape torch.Size([32, 64, 64]) from checkpoint, the shape in current model is torch.Size([16, 64, 64]). size mismatch for decode.18.weight: copying a param with shape torch.Size([32, 16, 5, 5]) from checkpoint, the shape in current model is torch.Size([1, 16, 7, 7]). size mismatch for decode.18.bias: copying a param with shape torch.Size([16]) from checkpoint, the shape in current model is torch.Size([1]).

What is the problem and how can I fix it？Thank you！！

手顺

• 代码版本 : e3bd2e
• 最后更新时间: Date: Sat Jul 23 22:53:13 2022 +0800

使用命令将ttf/otf文件做成数据集

使用命令:

> cd data_utils

> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split train  
> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split test

> python write_glyph_imgs.py --split train  
> python write_glyph_imgs.py --split test  


> python write_data_to_dirs.py --split train
> python write_data_to_dirs.py --split test

步骤2:train the neural rasterizer:

使用命令:

• mv ./data/vecfont_dataset_dirs_/ ./data/vecfont_dataset_dirs
• python train_nr.py --mode train --experiment_name dvf --model_name neural_raster

发生下面错误:

Training on experiment dvf_neural_raster...
Finished loading train paths
Traceback (most recent call last):
  File "/home/ubuntu/deepvecfontnew/train_nr.py", line 231, in <module>
    main()
  File "/home/ubuntu/deepvecfontnew/train_nr.py", line 223, in main
    train(opts)
  File "/home/ubuntu/deepvecfontnew/train_nr.py", line 191, in train
    train_nr_model(opts)
  File "/home/ubuntu/deepvecfontnew/train_nr.py", line 29, in train_nr_model
    train_loader = get_loader(opts.data_root, opts.image_size, opts.char_categories, opts.max_seq_len, opts.seq_feature_dim, opts.batch_size, opts.read_mode, opts.mode)
  File "/home/ubuntu/deepvecfontnew/dataloader.py", line 69, in get_loader
    dataloader = data.DataLoader(dataset, batch_size, shuffle=(mode == 'train'), num_workers=batch_size, drop_last=True)
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 347, in __init__
    sampler = RandomSampler(dataset, generator=generator)  # type: ignore[arg-type]
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 107, in __init__
    raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0

修改代码

将文件 dataloader.py 69的shuffle=(mode == 'train')修改为shuffle=False 继续使用命令:python train_nr.py --mode train --experiment_name dvf --model_name neural_raster

在train_nr.py 的57行加入 print(len(train_loader))

输出一下log:
```
Training on experiment dvf_neural_raster...
Finished loading train paths
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py:557: 	UserWarning: This DataLoader will create 64 worker processes in total. Our suggested max number of 	worker in current system is 12, which is smaller than what this DataLoader is going to create. Please be 	aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker 	number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
Finished loading test paths
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torchvision/models/_utils.py:208: 	UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 	'weights' instead.
  warnings.warn(
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torchvision/models/_utils.py:223: 	UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and 	will be removed in 0.15. The current behavior is equivalent to passing `weights=VGG19_Weights.	IMAGENET1K_V1`. You can also use `weights=VGG19_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)

0

0
0
0
0
0
0
0
```

显示len(train_loader)的值为0?这是因为生成的数据有问题吗？还是因为我修改的代码导致的原因。

步骤3. train the image super-resolution model:

使用命令: python train_sr.py --mode train --name image_sr

发生下面错误:

----------------- Options ---------------
               batch_size: 16
                    beta1: 0.0
          char_categories: 52
          checkpoints_dir: ./experiments
           continue_train: False
                crop_size: 256
                 dataroot: ./data/vecfont_dataset_dirs_sr/
             dataset_mode: aligned
                direction: BtoA
              display_env: main
             display_freq: 400
               display_id: 1
            display_ncols: 4
             display_port: 8097
           display_server: http://172.31.222.102
          display_winsize: 256
                    epoch: latest
              epoch_count: 1
          experiment_name: dvf
                 gan_mode: lsgan
        gauss_temperature: 0
                  gpu_ids: 0
               image_size: 256
                   img_hr: 256
                   img_lr: 128
                init_gain: 0.02
                init_type: normal
                 input_nc: 1
                  isTrain: True                                 [default: None]
                lambda_L1: 1.0
                load_iter: 0                                    [default: 0]
                load_size: 256
                       lr: 0.002
           lr_decay_iters: 50
                lr_policy: linear
         max_dataset_size: inf
          mix_temperature: 0.0001
                     mode: train
               model_name: main_model
                 n_epochs: 500
           n_epochs_decay: 500
               n_layers_D: 3
                     name: image_sr
                      ndf: 64
                     netD: basic
                     netG: unet_256
                      ngf: 64
               no_dropout: False
                  no_flip: True
                  no_html: False
                     norm: instance
              num_threads: 4
                output_nc: 1
                    phase: train
                pool_size: 50
               preprocess: none
               print_freq: 100
                read_mode: dirs
             save_by_iter: False
          save_epoch_freq: 25
         save_latest_freq: 5000
           serial_batches: False
                   suffix:
               test_epoch: 125
         update_html_freq: 1000
                  verbose: False
----------------- End -------------------
Traceback (most recent call last):
  File "/home/ubuntu/deepvecfontnew/train_sr.py", line 12, in <module>
    dataset_train, dataset_test = create_dataset(opt)  # create a dataset given opt.dataset_mode and other options
  File "/home/ubuntu/deepvecfontnew/models/imgsr/modules.py", line 244, in create_dataset
    dataset_train = get_loader(opt.dataroot, opt.char_categories, opt.batch_size, opt.read_mode, opt.img_lr, opt.img_hr, 'train')
  File "/home/ubuntu/deepvecfontnew/dataloader_sr.py", line 63, in get_loader
    dataloader = data.DataLoader(dataset, batch_size, shuffle=(mode == 'train'), num_workers=batch_size)
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 347, in __init__
    sampler = RandomSampler(dataset, generator=generator)  # type: ignore[arg-type]
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/sampler.py", line 107, in __init__
    raise ValueError("num_samples should be a positive integer "
ValueError: num_samples should be a positive integer value, but got num_samples=0

修改代码解决错误

将dataloader_sr.py文件63行的shuffle=(mode == 'train')=> shuffle=False

再次运行步骤4发生以下错误:

home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/optim/lr_scheduler.py:131: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  	warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
	learning rate 0.0020000 -> 0.0020000
	End of epoch 1 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 2 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 3 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 4 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 5 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 6 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 7 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 8 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 9 / 1000    Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 10 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 11 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 12 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 13 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 14 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 15 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 16 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 17 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 18 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 19 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 20 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 21 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 22 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 23 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	End of epoch 24 / 1000   Time Taken: 0 sec
	learning rate 0.0020000 -> 0.0020000
	Traceback (most recent call last):
	  File "/home/ubuntu/deepvecfontnew/train_sr.py", line 45, in <module>
	    writer.add_scalar('TRAIN/l1_loss', criterionL1(model.real_B, model.fake_B), epoch)
	AttributeError: 'Pix2PixModel' object has no attribute 'real_B'

问题

• 步骤2中将文件 dataloader.py 69行的shuffle=(mode == 'train')修改为shuffle=False，是否会存在问题?
• 步骤2中在train_nr.py 的57行加入 print(len(train_loader))，输出的结果为0，是因为数据有问题吗？还是其他原因?
• 步骤3出现的错误AttributeError: 'Pix2PixModel' object has no attribute 'real_B'，这是什么原因?

手顺

代码版本 3ba4adb
最后更新时间: Fri Dec 31 12:10:07 2021 +0800

使用命令将ttf/otf文件做成数据集

使用命令:
> cd data_utils

> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split train  
> fontforge -lang=py -script convert_ttf_to_sfd_mp.py --split test

> python write_glyph_imgs.py --split train  
> python write_glyph_imgs.py --split test  
> python write_glyph_imgs.py --split train --img_size=256  
> python write_glyph_imgs.py --split test --img_size=256  

> python write_data_to_pkl.py --split train  
> python write_data_to_pkl.py --split test  

步骤2:train the neural rasterizer:

使用命令

mv ./data/vecfont_dataset_/ ./data/vecfont_dataset/
mv  ./data/vecfont_dataset/train/mean.npz ./data/vecfont_dataset/train/stdev.npz ./data
python train_nr.py --mode train --experiment_name dvf --model_name neural_raster

步骤3. train the image super-resolution model:

使用命令: cp -r ./data/vecfont_dataset ./data/glyphss_dataset python train_sr.py --mode train --name image_sr

出现下面错误

----------------- Options ---------------
               batch_size: 2
                    beta1: 0.0
          char_categories: 52
          checkpoints_dir: ./experiments
           continue_train: False
                crop_size: 256
                 dataroot: ./data/glyphss_dataset/
             dataset_mode: aligned
                direction: BtoA
              display_env: main
             display_freq: 400
               display_id: 1
            display_ncols: 4
             display_port: 8097
           display_server: http://172.31.222.102
          display_winsize: 256
                    epoch: latest
              epoch_count: 1
          experiment_name: dvf
                 gan_mode: lsgan
        gauss_temperature: 0
                  gpu_ids: 0
               image_size: 256
                init_gain: 0.02
                init_type: normal
                 input_nc: 1
                  isTrain: True                                 [default: None]
                lambda_L1: 1.0
                load_iter: 0                                    [default: 0]
                load_size: 256
                       lr: 0.002
           lr_decay_iters: 50
                lr_policy: linear
         max_dataset_size: inf
          mix_temperature: 0.0001
                     mode: train
               model_name: main_model
                 n_epochs: 500
           n_epochs_decay: 500
               n_layers_D: 3
                     name: image_sr
                      ndf: 64
                     netD: basic
                     netG: unet_256
                      ngf: 64
               no_dropout: False
                  no_flip: True
                  no_html: False
                     norm: instance
              num_threads: 4
                output_nc: 1
                    phase: train
                pool_size: 50
               preprocess: none
               print_freq: 100
             save_by_iter: False
          save_epoch_freq: 25
         save_latest_freq: 5000
           serial_batches: False
                   suffix:
               test_epoch: 125
         update_html_freq: 1000
                  verbose: False
----------------- End -------------------
Loading ./data/glyphss_dataset/train/train_all.pkl pickle file ...
Finished loading
Loading ./data/glyphss_dataset/test/test_all.pkl pickle file ...
Finished loading

initialize network with normal
initialize network with normal
model [Pix2PixModel] was created
---------- Networks initialized -------------
[Network G] Total number of parameters : 54.403 M
[Network D] Total number of parameters : 2.764 M
-----------------------------------------------
/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/optim/lr_scheduler.py:131: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
learning rate 0.0020000 -> 0.0020000
Traceback (most recent call last):
  File "/home/ubuntu/deepvecfont/train_sr.py", line 29, in <module>
    for i, data in enumerate(dataset_train):  # inner loop within one epoch
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 652, in __next__
    data = self._next_data()
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1347, in _next_data
    return self._process_data(data)
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1373, in _process_data
    data.reraise()
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/_utils.py", line 461, in reraise
    raise exception
KeyError: Caught KeyError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ubuntu/anaconda3/envs/dvf/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 49, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/ubuntu/deepvecfont/dataloader_sr.py", line 30, in __getitem__
    item['rendered_256'] = torch.FloatTensor(cur_glyph['rendered_256']).view(self.char_num, 256, 256) / 255.
KeyError: 'rendered_256'

修改错误

在data_util/write_data_to_pkl.py文件的93行追加以下代码,解决错误

```
93                 if not os.path.exists(os.path.join(cur_font_sfd_dir, 'imgs_256.npy')):
94                     rendered256 = np.zeros((opts.num_char, opts.img_size, opts.img_size), np.uint8)
95                     rendered256[:, :, :] = 255
96                     rendered256 = rendered.tolist()
97                 else:
98                     rendered256 = np.load(os.path.join(cur_font_sfd_dir, 'imgs_256.npy')).tolist()
99                 merged_res['rendered_256'] = rendered256

```

步骤4: train our main model

python main.py --mode train --experiment_name dvf --model_name main_model

发生错误:

Epoch: 899/2000, Batch: 0/1, Loss: 0.913249, img_l1_loss: 0.168880, kl_loss: 0.003375, img_pt_c_loss: 0.020982, mdn_loss: 0.461596, softmax_xent_loss: 0.073407, synsvg_nr_recloss: 0.185008
Epoch: 949/2000, Batch: 0/1, Loss: 0.678629, img_l1_loss: 0.123693, kl_loss: 0.006045, img_pt_c_loss: 0.020730, mdn_loss: 0.283389, softmax_xent_loss: 0.115898, synsvg_nr_recloss: 0.128875
Epoch: 999/2000, Batch: 0/1, Loss: 0.780886, img_l1_loss: 0.189140, kl_loss: 0.004557, img_pt_c_loss: 0.024209, mdn_loss: 0.270104, softmax_xent_loss: 0.115126, synsvg_nr_recloss: 0.177751
Traceback (most recent call last):
  File "/home/ubuntu/deepvecfont/main.py", line 358, in <module>
    main()
  File "/home/ubuntu/deepvecfont/main.py", line 352, in main
    train(opts)
  File "/home/ubuntu/deepvecfont/main.py", line 327, in train
    train_main_model(opts)
  File "/home/ubuntu/deepvecfont/main.py", line 167, in train_main_model
    val_img_decoder_out, val_vggpt_loss, val_kl_loss, val_svg_losses, val_trg_img, val_ref_img, val_trgsvg_nr_out, val_synsvg_nr_out = network_forward(val_data, mean, std, opts, network_modules)
  File "/home/ubuntu/deepvecfont/main.py", line 318, in network_forward
    svg_losses = mdn_top_layer.svg_loss(top_output, trg_seq, trg_seqlen+1, opts.max_seq_len)
  File "/home/ubuntu/deepvecfont/models/svg_decoder.py", line 185, in svg_loss
    seqlen_mask = util_funcs.sequence_mask(trg_seqlen, max_seq_len)
  File "/home/ubuntu/deepvecfont/models/util_funcs.py", line 80, in sequence_mask
    .lt(lengths.unsqueeze(1)))
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

问题

• 步骤2中发生了错误, 我修改了data_util/write_data_to_pkl.py中的代码，追加代码如下图所示，这样修改代码是否存在问题?

• 步骤4中发生的错误，是做成的数据有问题吗？还是其他原因?可以帮忙看一下吗？

1. When I am training the main module, I get OSError: [WinError 1455] and memory error, can you please provide me with what I need to train the model?
2. Is the current program input data only supports svg, does it support bitmap?
3. What directory should Input glyphs be placed in the demo, what is the program run command, and in which directory will the fonts be generated by deepvecfont