Language models are open knowledge graphs ( non official implementation )

Overview

language-models-are-knowledge-graphs-pytorch

Language models are open knowledge graphs ( work in progress )

A non official reimplementation of Language models are open knowledge graphs

The implemtation of Match is in process.py

example bob dylan

Execute MAMA(Match and Map) section

Do note the extracted results is still quite noisy and should then filtered based on relation unique pair frequency

python extract.py examples/bob_dylan.txt bert-large-cased-bob_dynlan.jsonl --language_model bert-large-cased --use_cuda true

Map

  1. Entity linking

The original download link for Stanford Entity linking is removed (nlp.stanford.edu/pubs/crosswikis-data.tar.bz2)[nlp.stanford.edu/pubs/crosswikis-data.tar.bz2]. I will use (REL)[https://github.com/informagi/REL] for entity disambiguation model (supervised instead of the original unsupervied) to achieve the same task.

  1. Relations linking (page 5, 2.2.1)

Lemmatization is done in the previous steps process.py, in this stage we remove inflection, auxiliary verbs, adjectives, adverbs words.

Adjectives extracted from here: https://gist.github.com/hugsy/8910dc78d208e40de42deb29e62df913

Adverbs extracted from here : https://raw.githubusercontent.com/janester/mad_libs/master/List%20of%20Adverbs.txt

Environment setup

This repo is run using virtualenv

virtualenv -p python3 env
source env/bin/activate
pip install -r requirements.txt
Comments
  • Issues with installation and running extract

    Issues with installation and running extract

    The install links in requirements.txt for spacy models dont work. I had to manually install them.

    Further, after installing the models, I keep getting the following error : Traceback (most recent call last): File "language-models-are-knowledge-graphs-pytorch-main/extract.py", line 3, in from mapper import Map, deduplication File "language-models-are-knowledge-graphs-pytorch-main/mapper.py", line 3, in from REL.db.generic import GenericLookup ModuleNotFoundError: No module named 'REL'

    I tried installing rel which further throws the following error : pip install rel

    Looking in indexes: https://pypi.org/simple Collecting rel Using cached rel-0.3.1.tar.gz (9.0 kB) ERROR: Command errored out with exit status 1: command: language-models-are-knowledge-graphs-pytorch-main/env/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/setup.py'"'"'; file='"'"'/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-pip-egg-info-_ir2rrqc cwd: /private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/ Complete output (7 lines): Traceback (most recent call last): File "", line 1, in File "/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/setup.py", line 1, in from rel import version File "/private/var/folders/lg/cdx1ddsd2yz791mb004lbpg8dxjqm5/T/pip-install-07fd7ph9/rel_54cb449f7f8d4ed8871ce0cb5c419cc6/rel/init.py", line 2, in from rel import override, supported_methods, initialize, read, write, timeout, signal, event, dispatch, loop, abort, init, sys, EV_PERSIST, EV_READ, EV_SIGNAL, EV_TIMEOUT, EV_WRITE ImportError: cannot import name 'override' ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

    Any help is appreciated.

    opened by nidharap 9
  • sqlite3.OperationalError: unable to open database file

    sqlite3.OperationalError: unable to open database file

    Hi, python extract.py examples/bob_dylan.txt bert-large-cased-bob_dynlan.jsonl --language_model bert-large-cased --use_cuda true wandb: WARNING W&B installed but not logged in. Run wandb login or set the WANDB_API_KEY env variable. Traceback (most recent call last): File "extract.py", line 3, in from mapper import Map, deduplication File "language-models-are-knowledge-graphs-pytorch/mapper.py", line 5, in emb = GenericLookup("entity_word_embedding", save_dir=sqlite_path, table_name="embeddings") File "anaconda3/lib/python3.6/site-packages/REL/db/generic.py", line 36, in init self.db = self.initialize_db(path_db, table_name, columns) File "anaconda3/lib/python3.6/site-packages/REL/db/base.py", line 41, in initialize_db db = sqlite3.connect(fname, isolation_level=None) sqlite3.OperationalError: unable to open database file

    thanks

    opened by molyswu 1
  • Issue with using REL

    Issue with using REL

    Hey,

    I get the following error while running the extract.py file.

    Traceback (most recent call last): File "extract.py", line 88, in mapped_triplet = Map(head, relations, tail) File "/Users/I535445/Desktop/Me/new/mapper.py", line 12, in Map head_p_e_m = emb.wiki(str(head), 'wiki') File "/Users/I535445/Desktop/Me/new/env1/lib/python3.8/site-packages/REL/db/generic.py", line 45, in wiki g = self.lookup_wik(mention, table_name, column_name) File "/Users/I535445/Desktop/Me/new/env1/lib/python3.8/site-packages/REL/db/base.py", line 182, in lookup_wik e = c.execute( sqlite3.OperationalError: no such table: wiki

    Can someone please help ?

    opened by aqsakhan10 0
  • Using GPT as shown in the paper

    Using GPT as shown in the paper

    I am interested in using GPT as the main model as BERT is not performing too well.

    Has it been implemented? Otherwise I am ready to implement it myself. Please let me know

    opened by MikeDoes 3
  •  Hello, I am a beginner in Python

    Hello, I am a beginner in Python

    Hello, I am a beginner in Python. I am studying hard on the code that I really need. There are several python files (utils.py, constant.py, extract.py, mapper.py, stats,py). What order should they be executed? I would really appreciate it if you let me know

    opened by massai5 1
  • extract.py not running even with all requirements installed

    extract.py not running even with all requirements installed

    When running python extract.py examples/bob_dylan.txt bert-large-cased-bob_dynlan.jsonl --language_model bert-large-cased --use_cuda true

    on google collab the following error appears:

    /usr/local/lib/python3.7/dist-packages/spacy/util.py:275: UserWarning: [W031] Model 'en_core_web_sm' (2.2.5) requires spaCy v2.2 and is incompatible with the current spaCy version (2.3.2). This may lead to unexpected results or runtime errors. To resolve this, download a newer compatible model or retrain your custom model with the current spaCy version. For more details and available updates, run: python -m spacy validate warnings.warn(warn_msg) 2021-04-22 19:14:11.225481: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 Traceback (most recent call last): File "extract.py", line 3, in from mapper import Map, deduplication File "/content/mapper.py", line 5, in emb = GenericLookup("entity_word_embedding", save_dir=sqlite_path, table_name="embeddings") File "/usr/local/lib/python3.7/dist-packages/REL/db/generic.py", line 36, in init self.db = self.initialize_db(path_db, table_name, columns) File "/usr/local/lib/python3.7/dist-packages/REL/db/base.py", line 41, in initialize_db db = sqlite3.connect(fname, isolation_level=None) sqlite3.OperationalError: unable to open database file

    opened by connexionist 4
  • The issuues about BFS

    The issuues about BFS

    Your BFS function always return an array of length 3 because of your "visited[i] == False" I don't kown it‘s a bug or you do it on purpose. Actually when I read the Beam Search part of the origin paper,I feel confuse too. Look forward to your reply

    opened by jasou1995 0
  • function of black list relations (and issue with lemmatisation)

    function of black list relations (and issue with lemmatisation)

    Hi black cat,

    Thanks for the public implementation!

    I was wondering what the function of the blacklist relation is? Seems like it removed the option that noun_chunk can be in the path, but I see no mention of this in the original paper and I also see that the underlying assumption (that noun chunk can't be relations) might be false. Was hoping you could share your perspective on this? Hopefully I am just missing a good point :)

    A sidenote on the lemmatization used in filtering is also that the lemmatization is done on a word by word basis, which is flawed as lemmatization is contextually dependent for instance the lemma of meeting can either be meeting (a meeting) or meet (to meet).

    Best and thanks again for the public implementation, Kenneth

    opened by KennethEnevoldsen 0
Owner
theblackcat102
switch between academia and dev: No fancy keywords here
theblackcat102
Implementation for the EMNLP 2021 paper "Interactive Machine Comprehension with Dynamic Knowledge Graphs".

Interactive Machine Comprehension with Dynamic Knowledge Graphs Implementation for the EMNLP 2021 paper. Dependencies apt-get -y update apt-get instal

Xingdi (Eric) Yuan 19 Aug 23, 2022
The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

Neural Deformation Graphs Project Page | Paper | Video Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction Aljaž Božič, Pablo P

Aljaz Bozic 134 Dec 16, 2022
A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection 1. 介绍 用以替代 NMS,在所有 bbox 中挑选出最优的集合。 NMS 仅考虑了 bbox 的得分,然后根据 IOU 来

null 44 Sep 15, 2022
A non-linear, non-parametric Machine Learning method capable of modeling complex datasets

Fast Symbolic Regression Symbolic Regression is a non-linear, non-parametric Machine Learning method capable of modeling complex data sets. fastsr aim

VAMSHI CHOWDARY 3 Jun 22, 2022
Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Networks

CyGNet This repository reproduces the AAAI'21 paper “Learning from History: Modeling Temporal Knowledge Graphs with Sequential Copy-Generation Network

CunchaoZ 89 Jan 3, 2023
ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs

(Comet-) ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs Paper Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sa

AI2 152 Dec 27, 2022
Continuous Query Decomposition for Complex Query Answering in Incomplete Knowledge Graphs

Continuous Query Decomposition This repository contains the official implementation for our ICLR 2021 (Oral) paper, Complex Query Answering with Neura

UCL Natural Language Processing 71 Dec 29, 2022
Compositional and Parameter-Efficient Representations for Large Knowledge Graphs

NodePiece - Compositional and Parameter-Efficient Representations for Large Knowledge Graphs NodePiece is a "tokenizer" for reducing entity vocabulary

Michael Galkin 107 Jan 4, 2023
Code for the paper "Query Embedding on Hyper-relational Knowledge Graphs"

Query Embedding on Hyper-Relational Knowledge Graphs This repository contains the code used for the experiments in the paper Query Embedding on Hyper-

DimitrisAlivas 19 Jul 26, 2022
Collective Multi-type Entity Alignment Between Knowledge Graphs (WWW'20)

CG-MuAlign A reference implementation for "Collective Multi-type Entity Alignment Between Knowledge Graphs", published in WWW 2020. If you find our pa

Bran Zhu 28 Dec 11, 2022
ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representa

Bats Research 94 Nov 21, 2022
This is the source code for: Context-aware Entity Typing in Knowledge Graphs.

This is the source code for: Context-aware Entity Typing in Knowledge Graphs.

null 9 Sep 1, 2022
ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs

ConE: Cone Embeddings for Multi-Hop Reasoning over Knowledge Graphs This is the code of paper ConE: Cone Embeddings for Multi-Hop Reasoning over Knowl

MIRA Lab 33 Dec 7, 2022
TF2 implementation of knowledge distillation using the "function matching" hypothesis from the paper Knowledge distillation: A good teacher is patient and consistent by Beyer et al.

FunMatch-Distillation TF2 implementation of knowledge distillation using the "function matching" hypothesis from the paper Knowledge distillation: A g

Sayak Paul 67 Dec 20, 2022
source code for 'Finding Valid Adjustments under Non-ignorability with Minimal DAG Knowledge' by A. Shah, K. Shanmugam, K. Ahuja

Source code for "Finding Valid Adjustments under Non-ignorability with Minimal DAG Knowledge" Reference: Abhin Shah, Karthikeyan Shanmugam, Kartik Ahu

Abhin Shah 1 Jun 3, 2022
Using pretrained language models for biomedical knowledge graph completion.

LMs for biomedical KG completion This repository contains code to run the experiments described in: Scientific Language Models for Biomedical Knowledg

Rahul Nadkarni 41 Nov 30, 2022
Source code for paper: Knowledge Inheritance for Pre-trained Language Models

Knowledge-Inheritance Source code paper: Knowledge Inheritance for Pre-trained Language Models (preprint). The trained model parameters (in Fairseq fo

THUNLP 31 Nov 19, 2022
[IJCAI-2021] A benchmark of data-free knowledge distillation from paper "Contrastive Model Inversion for Data-Free Knowledge Distillation"

DataFree A benchmark of data-free knowledge distillation from paper "Contrastive Model Inversion for Data-Free Knowledge Distillation" Authors: Gongfa

ZJU-VIPA 47 Jan 9, 2023
Source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated Recurrent Memory Network

KaGRMN-DSG_ABSA This repository contains the PyTorch source Code for our paper: Understand me, if you refer to Aspect Knowledge: Knowledge-aware Gated

XingBowen 4 May 20, 2022