你好，我发现K-bert目前的entity linking的方式比较直接，并不会对有多种含义的实体进行区别，在实际使用过程中这往往会带来噪声，特别是一些大的知识库，有高频词带低频标签的情况。 比如用到过一个百科知识库里，有 舒适 → 作家名 这样的实体映射，也有大量一词多义的现象，单纯从word matching选择实体很容易将大量的噪声带入。 在实际使用中我发现特别是inference过程，大量是因为这类噪声知识的引入，反而让模型错判。

1 针对目前直接查表的entity linking，请问有没有合适的降噪思路？ 2 我看到作者提到“知识驱动”型任务，在微观层面是对每个识别的实体判断是否需要引入知识，请问这方面有没有什么思路或者可以借鉴的前沿研究可以参考？ 比如早年比较火的attention机制，有没有可能引入类似selective attention的方法，来判断实体是否应该挂载知识以及挂载哪个知识，这类思路和k-bert的知识挂载方式有没有结合点？

Dear @autoliuweijie,

thank you so much for your study. I would like to experiment K-BERT for a custom graphs. However, I need to convert to spo format. I checked the examples, which are in Chinese. So I couldn't understand the contents and format of spo. I would appreciate it if you could give me an example in English.

Kind regards Ipek

您好，感谢您开源代码，我使用您的代码和您分享的book_review数据集进行试验，但是准确率只有79%，比您的结果低了快10个百分点。想问问您有没有什么建议呢？ 我的运行指令为：nohup python3 -u run_kbert_cls.py --train_path /data/book_review/train.tsv --dev_path /data/book_review/dev.tsv --test_path /data/book_review/test.tsv --epochs_num 5 --batch_size 32 --kg_name CnDbpedia --output_model_path ./outputs/kbert_bookreview_CnDbpedia.bin > ./outputs/kbert_bookreview_CnDbpedia.log & (谷歌预训练模型和词表等文件路径我在程序中指定，max_length是默认的256） 我的结果为：79%

Hi, I am interested in this paper, while I can not find the NLPCC-DBQA dataset from your source codes. Could you show the results of the NLPCC-DBQA from your codes? Thanks.

Dear Dr.Liu:
I am a master's student at ISCAS, My research is focused on EMR information extraction. I have recently read your paper "K-BERT: Enabling Language Representation with Knowledge Graph" and got a lot of inspiration from it, thank you! I am wondering if you could kindly send me the slides for presentation or other materials about it. I promise they will be used only for research purposed. My email：[email protected] Yours sincerely, Wenwen Xu

Vocabulary file line 344 has bad format token Vocabulary Size: 21128 [BertClassifier] use visible_matrix: True [KnowledgeGraph] Loading spo from /home/schen/K-BERT/brain/kgs/Medical.spo Start training. Loading sentences from ./datasets/medical_ner/train.tsv There are 6919 sentence in total. We use 1 processes to inject knowledge into sentences. {'text_a': 0, 'label': 1} Progress of process 0: 0/6919 ['山 ， 男 ， 7 3 岁 ， 汉 族 ， 已 婚 ， 现 住 双 滦 区 陈 栅 子 乡 太 阳 沟 村 。', 'O O O O O O O O O O O O O O O O O O O O O O O O O O O O']

Traceback (most recent call last): File "run_kbert_cls.py", line 582, in main() File "run_kbert_cls.py", line 501, in main trainset = read_dataset(args.train_path, workers_num=args.workers_num) File "run_kbert_cls.py", line 329, in read_dataset dataset = add_knowledge_worker(params) File "run_kbert_cls.py", line 84, in add_knowledge_worker label = int(line[columns["label"]]) ValueError: invalid literal for int() with base 10: 'O O O O O O O O O O O O O O O O O O O O O O O O O O O O'

Hello. I run the model with my custom dataset. However, it returns "ZeroDivisionError" as follows. What the most possible troubleshoot for this? Is there something wrong with the dataset? Thank you.

Hello, How are you enforcing the -inf condition if the two words are not in the same branch ? In the code you are setting both the places as 1, bit shouldn't it be 0 and -inf ?

Calculate Visible Matrix

        visible_matrix = np.zeros((token_num,token_num))
        for item in abs_idx_tree:
            src_ids = item[0]
            for id in src_ids:
                visible_abs_idx = abs_idx_src + [idx for ent in item[1] for idx in ent]
                #print(visible_abs_idx)
                visible_matrix[id,visible_abs_idx] = 1 
            for ent in item[1]:
                for id in ent:
                    visible_abs_idx = ent + src_ids
                    visible_matrix[id,visible_abs_idx] = 1

您好，首先很感谢您对代码的分享！近期我在研究使用知识图谱进行文本分类的工作，在使用您的模型时，我发现三分类任务的性能相比二分类差距较大，因此想尝试更换更大的模型如bert-large。我想咨询一下，是不是下载bert-large之后下载uer/bert/bert-large-config就可以使用了？如果有必要的话，我还需要执行哪些操作以实现我的目的？

Hello @autoliuweijie, thank you for your amazing and inspiring work!

I would like to pre-train a K-Bert model on an english language corpus and to make it work I am currently trying to get the function in train_and_validate() to run, with args.target set to "bert". I notice that with this setting, BertDataLoader will be used for loading the data, but I am not sure what exact format the dataset file at dataset_path has to be. From the code, I see that it has to be pickle file, but I am having trouble trying to reconstruct one that works with the data loader.

It would be very helpful to have access to the data file originally used for pre-training. Could you provide a link or instructions on how to construct it myself?

您好，我正在复现您的代码，请问对文本分类任务是只能实现二分类还是可以实现多分类呢。我用自己的多类数据训练时报错： Traceback (most recent call last): File "run_kbert_cls.py", line 578, in main() File "run_kbert_cls.py", line 557, in main result = evaluate(args, False) File "run_kbert_cls.py", line 393, in evaluate print("Acc. (Correct/Total): {:.4f} ({}/{}) ".format(correct/len(dataset), correct, len(dataset))) ZeroDivisionError: division by zero