@backtime92 clovaai just released the LinkRefiner code clovaai/CRAFT-pytorch@3cd65f5 better you implement it, along with option to train . So that we can detect text-lines

@backtime92 Thanks for updating the code. I have a few questions.

1. Why is this function necessary to make the size of the character box 1.5 times larger by using enlargebox?

https://github.com/backtime92/CRAFT-Reimplementation/blob/fbaa63aebd61c2b290752102cdef4758891b1fb7/gaussianMap/gaussian.py#L104

2. What is the reason for shifting the box using the variable top_left? https://github.com/backtime92/CRAFT-Reimplementation/blob/fbaa63aebd61c2b290752102cdef4758891b1fb7/gaussianMap/gaussian.py#L113-L115

I ported your code to mxnet but when try to trained a model syntext. Gaussian map have some problem with my model.Right of page is lose. @backtime92 can you give some advice ?

I trained with the Synthtext data, and the loss became nan values ​​over 20,000 epochs. Can anyone give me advice on this issue?

3000 0.1077 4000 0.1032 5000 0.1003 6000 0.0984 7000 0.0970 8000 0.0956 9000 0.0948 10000 0.0940 11000 0.0933 12000 0.0925 13000 0.0919 14000 0.0914 15000 0.0909 16000 0.0904 17000 0.0900 18000 0.0897 19000 0.0893 20000 0.0889 21000 0.0884 22000 0.0878 23000 nan 24000 nan 25000 nan 26000 nan 27000 nan 28000 nan 29000 nan 30000 nan 31000 nan 32000 nan 33000 nan 34000 nan 35000 nan 36000 nan 37000 nan 38000 nan 39000 nan 40000 nan 41000 nan 42000 nan 43000 nan 44000 nan 45000 nan 46000 nan 47000 nan 48000 nan 49000 nan 50000 nan

Thanks for the code and model disclosure.

Test results performed with the IC15 pretrained model are different from those recorded.

I downloaded your ic15 pretrained model from Google Drive. After executing test function, evaluation function was executed.

In 'readme', recall = 76%, precision = 84%, hmean = 80% for ICDAR2015 dataset. But the evaluation result I did with the model is recall = 70.5%, precision = 80.3%, hmean = 75.1%.

Why are the results different? Is the model that measured the score different from the model of google drive?

~~How is Syndata.pth different from vgg16_bn-6c64b313.pth? My guess is vgg16_bn-6c64b313.pth is just fro vgg16 and Syndata.pth is for the rest?~~

when I use Syndata.pth in vgg16_bn.py, I get error: load_state_dict Missing keys ... but when I use vgg16_bn-6c64b313.pth, the model load properly.

1. is Syndata.pth for the same model as vgg16_bn-6c64b313.pth?
2. Why am I getting this bug?

Can you please let me if boundary box sorting is done? if not can you please help me for the same as i am working on something and want to get the boundary box sorted . I really appreciate.

Hi, I am trying to train the ICDAR2015 dataset. Thanks for your work! I found the training speed is quite slow. The problem exists in the data loader (the num of workers is set to 0). However, if I set it larger than 1, it would meet the below problem when generating the pseudo label.

  File "/home/craft_reimplementation/data_loader.py", line 125, in inference_pursedo_bboxes
    img_torch = img_torch.type(torch.FloatTensor).cuda()
RuntimeError: CUDA error: initialization error

Do you have any good idea to solve this issue?

Hi author, thanks for your open source.

When I train the icdar15 dataset, I met this problem:

"operands could not be broadcast together with shapes (512,512) (306,220). "

Do you also met this problem? And do you know which line of code causes this warning?

I experimented with the updated trainSynth.py, and there was no significant difference in performance from the code before the update.

Is there any difference between the updated code and before it is different except for the difference in gaussian map generation, transform, and augmentation(update code only random crop)?

And is the performance difference similar only in my experimental results?

The fig5 of the paper shows the results of adjustment for each epoch. do we have to make Pseudo-label for each epoch in terms of implementation, or do we create them with Synthetic datasets learned models and then not reflect them in the loss?

Found a small bug: On traversing through same image in multiple epochs and hence doing random_scaling every time, the coordinates in the previous epch get mutliplied by the scale factor every time, thus the dimensions of the character boxes overshoots the image dimension and hence no bounding boxes or region or affinity boxes made for that epoch.

Reason : The list indexing and slicing done in load_image_gt_and_confidencemask function in Synth80k class doesn't make a deep copy of the charbox coordinates and hence the same coordinates get modified.

Solution : using deepcopy for making list copy at line

_charbox = copy.deepcopy(self.charbox[index]).transpose((2, 1, 0))

https://github.com/backtime92/CRAFT-Reimplementation/blob/fbaa63aebd61c2b290752102cdef4758891b1fb7/data/dataset.py#L39

Although I found it working on master branch: https://github.com/backtime92/CRAFT-Reimplementation/blob/d24fd6895fa7153506f1114afc3ced99f07f889a/data_loader.py#L406

As a result in say 10th epoch, all we have is this image as output with no character boxes:

After fixing with deepcopy, then it produces this after 10 epochs and random scaling :

@backtime92 . Pardon me for tagging here but if this is not fixed already(excuse me if that is already fixed), this can certainly increase accuracy since if this is not done, then in the later epochs, all that the network is seeing is a plain image with no input character regions.

Hello in part 3.2.2 of the main paper related to this CRAFT-Reimplementation, the authors say : " When a real image with word-level annotations is provided, the learned interim model predicts the character region score of the cropped word images to generate character-level bounding boxes" As far as I realized this model is used for generating character-level bounding boxes for word images that do not have character annotation. My question is what is the interim model architecture and how is this model trained? Is the interim model trained with the cropped words in the synth-text images?

Hello. I experimented with the icdar2015 dataset by referring to the code you shared. The results of my experiments are not good(ICDAR2015 f1-score 0.78). Do you have any important tips for improving the performance?