Knowledge Base Completion Meets Transfer Learning
This code accompanies the paper Knowledge Base Completion Meets Transfer Learning published at EMNLP 2021.
Setup
Following packages are needed to run the code
- Python >=3.6
- pytorch>=1.6
- spacy>=2.0 and en_core_web_sm model
- tqdm
Run setup.sh to download and transform GloVe embeddings and OlpBench dataset. Please note that this downloads 3.5GB of files which unzip into around 10GB of content.
Running the code
For full help, run python main.py -h, a couple of examples are given below:
For pre-training TuckER on OlpBench, run
python main.py -data Data/OlpBench -dim 300 -lr 1e-4 -batch 4096 -n_epochs 100 -embedding TuckER -dropout 0.3 -encoder GRU -hits [1,3,5,10,30,50] -output_dir TuckEROlpBench300 -dump_vocab -only_batch_negative
For pretraining, it is important to add -dump_vocab to store encoder vocabulary. Otherwise it is not possible to load the stored model for fine-tuning. For any large-scale pre-training it is important to add -only_batch_negative argument to avoid encoding all entities at every training step.
To fine-tune the model obtained with the above command on ReVerb20K using NoEncoder, use the command below.
python main.py -data Data/ReVerb20K -dim 300 -lr 3e-4 -batch 512 -n_epochs 500 -embedding TuckER -dropout 0.3 -encoder NoEncoder -hits [1,3,5,10,30,50] -output_dir TuckERReVerb20K -pretrained_dir TuckEROlpBench300
To train the with same setup but from a randomly-initialized model, just remove the -pretrained_dir argument.
python main.py -data Data/ReVerb20K -dim 300 -lr 3e-4 -batch 512 -n_epochs 500 -embedding TuckER -dropout 0.3 -encoder NoEncoder -hits [1,3,5,10,30,50] -output_dir TuckERReVerb20K
Reference
If you use the code from this repo, please cite the following work.
@inproceedings{kocijan2021KBCtransfer,
title = "Knowledge Base Completion Meets Transfer Learning",
author = "Kocijan, Vid and
Lukasiewicz, Thomas",
booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
month = nov,
year = "2021",
address = "Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
}
45 Oct 7, 2022
126 Nov 22, 2022
3 Jan 3, 2023
18 Oct 6, 2022
66 Dec 26, 2022
29 Dec 28, 2022
73 Nov 6, 2022
1 Oct 26, 2021
32 Nov 9, 2021
.So I used the same hyperparameter training in the appendix(-data Data/ReVerb20K -dim 300 -lr 3e-3 -batch 1024 -embedding TuckER -dropout 0.4 -encoder GRU).
This is my best result for the test set. Could you tell me what the problem is? Thank you!
02/21/2022 15:59:36 - INFO - main - MR: Head 241.96 Tail 1240.93 Overall 741.45
02/21/2022 15:59:36 - INFO - main - MRR: Head 0.533 Tail 0.153 Overall 0.343
02/21/2022 15:59:36 - INFO - main - H@N 1: 0.277 3: 0.370 5: 0.410 10: 0.469 30: 0.558 50: 0.599
41 Nov 8, 2022
3.1k Jan 1, 2023
135 Oct 27, 2022
5 Dec 21, 2021
4 Nov 10, 2022
69 Nov 26, 2022
77 Nov 30, 2022
2 Oct 14, 2021
48 Dec 23, 2022
24 Aug 7, 2021