Hi, first of all thanks for the code, that's a great contribution to the community!!. From the paper I understood that the model could be fine-tuned end2end for EE and EL at the same time, however looking at the code I think it does not do it like that, right? Is it supported the combined EE and EL end2end somehow?

Thanks,

Thank you for the amazing work! I am training the model with a customized dataset. However, I just noticed after 5 epochs training, the learning rate came to 0 which makes model hard to learn. Could you please point me to the learning rate strategy of BROS and may I know how to change it according to my case? Thanks!

TRAIN [epoch: 0/50] || train_loss: 460.69653 || lr: 4e-05 || time: 193.6 secs. precision: 0.9080, recall: 0.9023, f1: 0.9052 TRAIN [epoch: 1/50] || train_loss: 129.8502 || lr: 3e-05 || time: 198.6 secs. precision: 0.9374, recall: 0.9184, f1: 0.9278 TRAIN [epoch: 2/50] || train_loss: 75.951 || lr: 3e-05 || time: 198.0 secs. precision: 0.9293, recall: 0.9183, f1: 0.9237 TRAIN [epoch: 3/50] || train_loss: 46.87292 || lr: 2e-05 || time: 197.8 secs. precision: 0.9442, recall: 0.9391, f1: 0.9416 TRAIN [epoch: 4/50] || train_loss: 28.64673 || lr: 1e-05 || time: 197.7 secs. precision: 0.9444, recall: 0.9392, f1: 0.9418 TRAIN [epoch: 5/50] || train_loss: 16.82515 || lr: 0.0 || time: 197.6 secs.

Hi. I've seen that the model has been uploaded to the huggingface hub but without any information in the card: https://huggingface.co/naver-clova-ocr/bros-base-uncased Just wondering when you are planning to upload the code in this repo. Thanks in advance,

RuntimeError: CUDA error: CUBLAS_STATUS_EXECUTION_FAILED when calling cublasGemmEx( handle, opa, opb, m, n, k, &falpha, a, CUDA_R_16F, lda, b, CUDA_R_16F, ldb, &fbeta, c, CUDA_R_16F, ldc, CUDA_R_32F, CUBLAS_GEMM_DFALT_TENSOR_OP)

This error while training the model CUDA_VISIBLE_DEVICES=0 python3 train.py --config=configs/custom.yaml

Thanks for the excellent work! I am trying to reproduce the result on CORD dataset. However, I find the f-score results in your paper are somewhat different from that in LayoutLMv2 paper. Specifically, LayoutLMv2*-base achieves 96.05 and LayoutLMv2*-large achieves 97.24 in your paper. While in LayoutLMv2 paper, LayoutLMv2-base achieves 94.95 and LayoutLMv2-large achieves 96.01. Could you give an example of BROS fine-tuning on CORD dataset? Thanks!

Hello,

I am trying to run the fine tuning scripts for FUNSD on Google Colab; I have installed all the required dependencies in requirements.txt, but when running

!CUDA_VISIBLE_DEVICES=0 python train.py --config=configs/finetune_funsd_ee_bies.yaml

I am getting

OSError: /usr/local/lib/python3.8/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN5torch6detail10class_baseC2ERKSsS3_SsRKSt9type_infoS6_

I have tried installing torchtext and upgraded pytorch lightning correspondingly, to no avail.

Any ideas on what could be going on?

Thanks!

Thanks for contributing this awesome piece of research!

Quick question about the input boxes.

1. For bros, is the expected format [x1, y1, x2, y2, x3, y3, x4, y4], where each x,y pair is the corners of the bounding box, starting from the top left and clockwise?

2. Each bounding box should be normalized by dividing x values by width and y values by height?

I'm training on DocVQA, but the results are not that great. Just trying to make sure I'm doing everything right :)

Hello, I am interested in the great work. However, I am a little bit confused about the linking task in CORD. Is the entity with category as "menu.nm" linking to all the other entities within a same group? Besides, do you use "is_key" to split a valid line (often in the bottom of an image) into 2 entities and then generate a link between them?

Hi there, first of all thanks for sharing your excellent work. I have a doubt regarding how you get the results of table 5. In the paper you mention that you don't use the order information, but how do you implement that exactly?. Do you remove the 1D abs. positional embeddings from the model? if so, that comes with a new pre-training? and finally, I guess you still train with the dataset order of the words and it is only in test where you shuffle the words, is that right?

Thanks in advance!

Could you guys please clarify whether num_samples_per_epoch in the config files refers to the total number of documents in the training set or does it mean something else?

I set the num_samples_per_epoch to the number of docs in my training set, however, the LRScheduler warmup is not working as expected.