A pure pytorch implemented ocr project including text detection and recognition

@courao Thank you for your hard work,

• Will you release CRNN training code & documentation.
• Can you add the option to extract the detected lines of CTPN.

@courao I have been training for 1 day on the SROIE dataset., the loss is still 0.44 ! It works well on other datasets, but not the SROIE dataset? Am I doing something wrong? dataset download link

您好，我和前面的朋友遇见的问题一样，修改crnn_recognizer.py文件的第100行def init(self, model_path='/root/zjut/ocr.pytorch/checkpoints/CRNN.pth')。当我执行'python demo.py'命令出错，显示如下： Traceback (most recent call last): File "/root/.vscode-server/extensions/ms-python.python-2019.11.50794/pythonFiles/ptvsd_launcher.py", line 43, in main(ptvsdArgs) File "/root/.vscode-server/extensions/ms-python.python-2019.11.50794/pythonFiles/lib/python/old_ptvsd/ptvsd/main.py", line 432, in main run() File "/root/.vscode-server/extensions/ms-python.python-2019.11.50794/pythonFiles/lib/python/old_ptvsd/ptvsd/main.py", line 316, in run_file runpy.run_path(target, run_name='main') File "/root/anaconda3/lib/python3.6/runpy.py", line 263, in run_path pkg_name=pkg_name, script_name=fname) File "/root/anaconda3/lib/python3.6/runpy.py", line 96, in _run_module_code mod_name, mod_spec, pkg_name, script_name) File "/root/anaconda3/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/root/zjut/ocr.pytorch/demo.py", line 10, in from ocr import ocr File "/root/zjut/ocr.pytorch/ocr.py", line 6, in recognizer = PytorchOcr() File "/root/zjut/ocr.pytorch/recognize/crnn_recognizer.py", line 111, in init self.model.load_state_dict({k.replace('module.', ''): v for k, v in torch.load(model_path).items()}) File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 845, in load_state_dict self.class.name, "\n\t".join(error_msgs))) RuntimeError: Error(s) in loading state_dict for CRNN: Missing key(s) in state_dict: "conv1.weight", "conv1.bias", "conv2.weight", "conv2.bias", "conv3_1.weight", "conv3_1.bias", "bn3.weight", "bn3.bias", "bn3.running_mean", "bn3.running_var", "conv3_2.weight", "conv3_2.bias", "conv4_1.weight", "conv4_1.bias", "bn4.weight", "bn4.bias", "bn4.running_mean", "bn4.running_var", "conv4_2.weight", "conv4_2.bias", "conv5.weight", "conv5.bias", "bn5.weight", "bn5.bias", "bn5.running_mean", "bn5.running_var". Unexpected key(s) in state_dict: "cnn.conv0.weight", "cnn.conv0.bias", "cnn.conv1.weight", "cnn.conv1.bias", "cnn.conv2.weight", "cnn.conv2.bias", "cnn.batchnorm2.weight", "cnn.batchnorm2.bias", "cnn.batchnorm2.running_mean", "cnn.batchnorm2.running_var", "cnn.batchnorm2.num_batches_tracked", "cnn.conv3.weight", "cnn.conv3.bias", "cnn.conv4.weight", "cnn.conv4.bias", "cnn.batchnorm4.weight", "cnn.batchnorm4.bias", "cnn.batchnorm4.running_mean", "cnn.batchnorm4.running_var", "cnn.batchnorm4.num_batches_tracked", "cnn.conv5.weight", "cnn.conv5.bias", "cnn.conv6.weight", "cnn.conv6.bias", "cnn.batchnorm6.weight", "cnn.batchnorm6.bias", "cnn.batchnorm6.running_mean", "cnn.batchnorm6.running_var", "cnn.batchnorm6.num_batches_tracked". size mismatch for rnn.1.embedding.weight: copying a param with shape torch.Size([5997, 512]) from checkpoint, the shape in current model is torch.Size([5835, 512]). size mismatch for rnn.1.embedding.bias: copying a param with shape torch.Size([5997]) from checkpoint, the shape in current model is torch.Size([5835]). 其中CRNN.pth是您度盘所提供的。

C:\Users\Administrator\AppData\Local\Programs\Python\Python37\python.exe C:/Users/Administrator/Desktop/pytorch/ocr.pytorch-master/demo.py Traceback (most recent call last): ./test_result\test_images\t1.txt File "C:/Users/Administrator/Desktop/pytorch/ocr.pytorch-master/demo.py", line 29, in txt_f = open(txt_file, 'w') FileNotFoundError: [Errno 2] No such file or directory: './test_result\test_images\t1.txt'

Process finished with exit code 1

我看到demo.py运行之后会清空掉test_result文件夹里的内容，然后就报这个错误。。。 求up主帮忙看一下。。

Can you please upload the pretrained models to another site than pan.baidu? Most non chinese users can't download from there. Maybe https://www.mediafire.com/, https://www.4shared.com/, https://mega.nz/, googledrive or https://zippyshare.com/ Thank you very much.

Hi! I've made a pytorch-lightning implementation of ctpn, mainly by using your code. Pytorch-lightning has many nice features, such as training with tpus/multiple gpus by changing one line of code, 16-bit precision, works on cpu (nice for testing), automatic learning rate finder... Would you be open to a pull request? Link to fork here! I'm in the process of converting your CRNN to pytorch-lightning as well.

Here's the simplified training loop:

datamodule = ICDARDataModule(
        config.icdar17_mlt_img_dir,
        config.icdar17_mlt_gt_dir,
        batch_size=1,
        num_workers=config.num_workers,
        shuffle=True,
    )

len_train_dataset = len(datamodule.train_data)

model = CTPN_Model()

trainer = pl.Trainer(gpus=1, # number of gpus, 0 if you want to use cpu
                       max_epochs=max_epochs,
                       log_every_n_steps=1,
                       callbacks=[LoadCheckpoint(config.pretrained_weights),
                                  InitializeWeights(),
                                  LossAndCheckpointCallback(config, len_train_dataset)])

trainer.fit(model, datamodule)

ocr.pytorch/detect/ctpn_predict.py第43行， image = image.astype(np.float32) - config.IMAGE_MEAN 对这一步的操作的意义（是减均值吗？），以及config.IMAGE_MEAN取值的依据不懂。。。 我是刚入门机器视觉，问题也比较小白，有劳up主了。。

I have an excel of 10k crops containing two columns:

Column 1: Image_path (<path/abc.png>)

Column 2: Ground_Truth ()

data.csv looks like:

path | gt

C:/Users/1234/crop/ABC 07 07 2020_page1.png | 8 05 75 824.46Cr C:/Users/1234/crop/PQW 07 10 2020_page1.png | Time 11 42 23 C:/Users/1234/crop/XRE 08 10 2020_page1.png | Account No. 200000592 C:/Users/1234/crop/JKL 07 10 2020_page1.png | 1 00 00 00 000.00

Now, I need to use this input for the training. Shall I feed this in dataset.py first ? Then need to start training?.

Or if you can help me with steps.

I am trying to apply this model to my own images which are in English but I got many Chinese Characters because I am using Baidu pertaining weighs? so Is there any option to predict only in English or any other trained weights?

我尝试训练CRNN模型， 但是我得到的结果一直是：

Not Covering Char: ១ - 6113 Not Covering Char: ១ - 6113 Not Covering Char: ៩ - 6121 Not Covering Char: ៧ - 6119 Not Covering Char: ១ - 6113 Not Covering Char: ទ - 6033 Not Covering Char: ស - 6047 Not Covering Char: រ - 6042 error Train loss: 0.000000

Start val ~/image0-1.jpg ~/image0-1.jpg pred :—眯恂 target:គោគ្គនាមនិងនាមៈ កូល វន្ធសហា 0.0 ocr_acc: 0.000000

请问我该如何成功地训练CRNN模型呢？感谢您的解答。