Hello, I am trying to run the code on a graph I have built that is not included among the choices provided in the repo. The graph I am working with is bipartite with typed edges.

To run the code, I prepared the list of triplets to be split in train, valid and test sets.

Triplets are saved in .tsv files:

...
idx__518	id	2449.0
idx__519	id	2452.0
idx__523	id	2469.0
idx__531	id	2484.0
idx__532	id	2487.0
idx__533	id	2494.0
idx__549	id	2545.0
...

To run my dataset, I slightly modified the python scripts in the repo.

In preprocess_dataset.py I added the line datasets = ['mydata'] to read from the folder src/src_data/mydata. I was then able to run the script, which created and filled the folder data/mydata.

DATA_PATH: /content/ssl-relation-prediction/data
Preparing dataset mydata
2681 entities and 9 relations
creating filtering lists
Done processing!
1

In main.py, I modified the list of datasets by adding mydata so that the code wouldn't raise an exception.

Finally, I tried to run the code with the arguments specified in the readme:

! python src/main.py --dataset mydata --score_rel True --model ComplEx --rank 1000 --learning_rate 0.1 --batch_size 1000 --lmbda 0.05 --w_rel 4 --max_epochs 100

Unfortunately, at this point the code fails because there is no train.npy file in the data folder. I assume that the train.npy file should have been created by the preprocessing script, but for some reason that did not happen. The content of the data folder is the following:

total 1320
-rw-r--r-- 1 root root   40662 Dec  3 12:59 ent_id
-rw-r--r-- 1 root root      97 Dec  3 12:59 rel_id
-rw-r--r-- 1 root root   43023 Dec  3 12:59 test.tsv.pickle
-rw-r--r-- 1 root root 1083297 Dec  3 12:59 to_skip.pickle
-rw-r--r-- 1 root root  140727 Dec  3 12:59 train.tsv.pickle
-rw-r--r-- 1 root root   32391 Dec  3 12:59 valid.tsv.pickle

It's not clear to me how to run custom-made datasets from the readme. Could you help me with that?

Hi Yihong, thank you so much for sharing this awesome repo!!

I tried to run the CP model; training seems to work, but when the validation starts, it crashes:

Evaluate the split train
Evaluating the rhs
Traceback (most recent call last):
  File "/home/jean/kg/src/main.py", line 25, in <module>
    main()
  File "/home/jean/kg/src/main.py", line 20, in main
    engine.episode()
  File "/home/jean/kg/src/engines.py", line 329, in episode
    self.validation_step(epoch=epoch, model=self.model)
  File "/home/jean/kg/src/engines.py", line 260, in validation_step
    res_s = self.dataset.eval(
  File "/home/jean/kg/src/datasets.py", line 243, in eval
    metrics = model.get_metric_ogb(
  File "/home/jean/kg/src/models/__init__.py", line 164, in get_metric_ogb
    cands = self.get_candidates(
TypeError: get_candidates() got an unexpected keyword argument 'indices'

Maybe the get_candidates(...) method of the CP class needs to be re-written?

Hi, thanks for your great contribution with this work :-) Do you plan to make the trained entity + relation embeddings for your SOTA models available for download?

Tara

Hi, I am trying your code an I think there is something wrong about the validation step in the training loop.

That is, consider the line at https://github.com/facebookresearch/ssl-relation-prediction/blob/main/src/engines.py#L197. At this line, the value assigned to the variable split is actually the last value that was assigned in the previous loop for computing step-wise metrics over all the splits (L.192-195). The last value is in fact "test".

So I think that at the end the model is wrongly validated against the test set instead of the validation set.

Hi! This is OGB Team.

We recently released https://github.com/snap-stanford/ogb/releases/tag/1.3.4 which improves the MRR calculation. This should not change your result if your model makes different predictive scores for different triplets, but would penalize those models that give the same predictive scores for different triplets (which is not ideal for link prediction).

Could you update your ogb package to 1.3.4 and run your model again just to confirm everything stays the same (the dataset stays the same. only evaluator changed)? Just one seed should be enough. Thanks!

Good evening. I have a question , regarding your code for the ogb wikidata. I've been working on graph neural networks lately, but because it's something new to me I'm having difficulties. Could I find somewhere documentation for the code and more specifically for the methods.Thanks in advance.

This is pull request was created automatically because we noticed your project was missing a Code of Conduct file.

Code of Conduct files facilitate respectful and constructive communities by establishing expected behaviors for project contributors.

This PR was crafted with love by Facebook's Open Source Team.

This is pull request was created automatically because we noticed your project was missing a Contributing file.

CONTRIBUTING files explain how a developer can contribute to the project - which you should actively encourage.

This PR was crafted with love by Facebook's Open Source Team.