the data you provide is right?? when I use preprocessed_KAIROS this file ,I don't find the following items which used in dataloader: 'input_token_ids': input_token_ids, 'input_attn_mask': input_attn_mask, 'tgt_token_ids': tgt_token_ids, 'tgt_attn_mask': tgt_attn_mask, 'doc_key': doc_keys,

Hello, I really like this project! I think the TapKey model for event trigger detection is not included in the repository. I would like to use it. I could just be missing it as well. Let me know. Thanks so much!

In scripts/test_rams.sh, when in head eval mode, the input file "data/RAMS_1.0/data/test_head.jsonlines" is missing in downloaded rams dataset. Also found missing file of "data/RAMS_1.0/data/test_head_coref.jsonlines".

Could you upload the above two missing files? Thanks.

Hi, thanks for your nice work. When analyzing data, I find many documents are very long. I find in your code, the MAX_LENGTH is 424. However, in Figure 4 of the paper: the distance of many informative args is greater than MAX_LENGTH, so many args are not even inputted to the encoder?

Thanks for sharing the code!

I could successfully training model on the wikievents dataset. But got the error saying "'BartConstrainedGen' object has no attribute postprocess_next_token_scores" when run scripts/test_kairos.sh.

It seems postprocess_next_token_scores method is not include in the BartConstrainedGen class (src/genie/constrained_gen.py). Is there something missing in the code or any other cause ? Look forward to your reply!

Hi, I tried to follow scripts/train_kairos.sh and scripts/test_kairos.sh but only received low performance as follow:

Role identification: P: 16.88, R: 4.456, F: 7.18 Role: P: 15.58, R: 4.21, F: 6.63 Coref Role identification: P: 19.48, R: 5.26, F: 8.29 Coref Role: P: 15.58, R: 4.21, F: 6.63

Even I tried to have more epochs , I can only get F1 score around 10. Is there anything goes wrong?

By the way, I failed to download the ckpt you shared on s3 due to a network error, is there any other way to acquire these files?

Thanks.

Hi, could you just quickly summarize the steps, required to test the downloaded checkpoint on the WikiEvents dataset?

As I have observed, the WikiEvents dataset is actually referred to as KAIROS in some parts of the code - it also uses KAIROS data module which requires, for example, the test file to be located in preprocessed_KAIROS/test.jsonl.

I did the following steps:

1. I have downloaded the WikiEvents dataset from S3 and stored in at data/wikievents.
2. I have downloaded the checkpoints from S3 which are stored at checkpoints/Wikievents/ (note that the directory contains both epoch=1-v0.ckpt and epoch=2-v0.ckpt).
3. I had to add the "--coref_dir" argument to the scripts/test_KAIROS.sh as it is referring to some other (non-existing) directory by default.
4. The command for "train.py" is the following:

python train.py --model=constrained-gen --ckpt_name=WikiEvents-pred \
    --load_ckpt=checkpoints/WikiEvents/epoch=2-v0.ckpt \
    --dataset=KAIROS \
    --eval_only \
    --train_file=data/wikievents/train.jsonl \
    --val_file=data/wikievents/dev.jsonl \
    --test_file=data/wikievents/test.jsonl \
    --coref_dir=data/wikievents/coref \
    --train_batch_size=4 \
    --eval_batch_size=4 \
    --learning_rate=3e-5 \
    --accumulate_grad_batches=4 \
    --num_train_epochs=3

Note that this throws an error as it still tries to load the test_file from "preprocessed_KAIROS/test.jsonl". 5. Hoping to fix the issue, I have copied the data/wikievents/ to ./preprocessed_KAIROS/. Unfortunately, I get the following error:

  File "/home/patrik/gen-arg/src/genie/data.py", line 15, in my_collate                                                
    doc_keys = [ex['doc_key'] for ex in batch]                                                                         
  File "/home/patrik/gen-arg/src/genie/data.py", line 15, in <listcomp>                                                
    doc_keys = [ex['doc_key'] for ex in batch]                                                                         
KeyError: 'doc_key'

Do you maybe have an idea about what am I doing wrong?

Best, Patrik

Hi!

When executing $ aws s3 ls s3://gen-arg-data/checkpoints/ or $ aws s3 cp s3://gen-arg-data/checkpoints/ ./ --recursive, I get the following error:

"An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied".

I am not sure if the issue is on my side?

Since there are coreferential entity mentions in a document, what is the principle to decide which entity mention should be annotated as the argument of an event? Besides, I find entity coreference clusters stored in split.jsonlines, but I do not find the event coreference clusters. Where are they stored?

Hi, thanks for sharing code!

I would like to ask where the "restrict the vocabulary of words to Vc" mentioned in the paper is reflected in the code?

In the code, it seems that the token with the highest probability is taken as the result by default.

Hi, thank you for this amazing work. When we try to test the model with pipeline_scorer it requires tgr-pred-file. The question is how to get this file for kairos dataset?

Hi,

Thank you for sharing your amazing work! I need some clarification regarding the implementation of equation 4 mentioned in the paper. From the initial reading of the code, it appears to me that, type clarification statements weren't used in the implementation.

Am I missing anything? Any help would be much appreciated!

Thank you for the great work. Could you please share the pretrained class vectors and tagger checkpoints for the tagger, for example all_class_vec_KAIROS.pt? Also I cannot quite figure out how to reproduce zero-shot event extraction (paragraph 4.4 of your paper). What should be the process (command line with arguments) to extract a new event? Alternatively, if I add more events to the KAIROS ontology, how could I fine-tune for them? As I understand, the tagger checkpoint is created in train_tagger.py. Then I'd need to use it to classify documents and add event types to training/test/dev data. Are there scripts for that?

Hi. I'm dealing with a dataset that has several argument roles, and each role might have multiple arguments. For example, '<arg1>, <arg2>, <arg3>, <arg4>, <arg5> has participate in an military activity on <arg6> ... ' In this sentence, arg1 to arg5 is the same role but different arguments, they are all the role 'countries'. And the problem is, each data in my dataset might have different number of 'countries'. In my example there are five countries which is arg1 to arg5, while in other data it might be just 3 countries which is arg1 to arg3 or just 1 countries. And I tried to use template like '<arg1>, <arg2>, <arg3>, <arg4>, <arg5> has participate in an military activity on <arg6> ... ', but I got pretty bad result, because it seems to be predicting a lot of '<arg> <arg> <arg>' and the result sentence looks absurd. If I use use template like '<arg1> has participate in an military activity on <arg2> ... ', the result is normal and acceptable, but in this way I can only predict a single one countries(arg1). Is there a way to deal with this? Thanks.

Love the paper.

I've tried it on my own closed domain dataset and achieved poor recall.

Role identification: P: 49.30, R: 28.43, F: 36.06
Role: P: 44.41, R: 25.60, F: 32.48
Coref Role identification: P: 69.93, R: 40.32, F: 51.15
Coref Role: P: 48.60, R: 28.02, F: 35.55

I believe the low recall is due to imbalanced labels, but I value recall over precision. Is there some way to tune the model to increase recall at the cost of precision?