hi,I run this code fluently，But the result is bad，I don't know what’s wrong with this. could you please give me some suggestions？thanks。

python -u run_scorer_test.py ./models/pairwise_scorers/test_events_average_0.8_model_3_corpus_level.conll events Processing file: ./models/pairwise_scorers/test_events_average_0.8_model_3_corpus_level.conll 0.65 1.0 0.787878787878788 0.8205128205128205 0.8205128205128205 0.8205128205128205 0.5474218379209004 0.6790832621385336 0.6061858323184838 0.15918234423790356 0.7040757533599581 0.2596591430831051 0.4649510356293238 0.6254223508155559 0.5333783332066294 {'mentions_recall': 0.65, 'mentions_precision': 1.0, 'mentions_f1': 0.787878787878788, 'muc_recall': 0.8205128205128205, 'muc_precision': 0.8205128205128205, 'muc_f1': 0.8205128205128205, 'bcub_recall': 0.5474218379209004, 'bcub_precision': 0.6790832621385336, 'bcub_f1': 0.6061858323184838, 'ceafe_recall': 0.15918234423790356, 'ceafe_precision': 0.7040757533599581, 'ceafe_f1': 0.2596591430831051, 'lea_recall': 0.4649510356293238, 'lea_precision': 0.6254223508155559, 'lea_f1': 0.5333783332066294, 'conll': 56.211926530480305}

What should I do about the predicted_topics_path entry in config_clustering.json? I have topic_level set to false, but predict.py still yields the following error.

python predict.py --config configs/config_clustering.json

gpu_num = [
  1
]
bert_model = "roberta-large"
hidden_layer = 1024
dropout = 0.3
with_mention_width = true
with_head_attention = true
embedding_dimension = 20
max_mention_span = 10
use_gold_mentions = false
mention_type = "mixed"
top_k = 0.25
split = "test"
training_method = "continue"
subtopic = false
use_predicted_topics = true
segment_window = 512
exact = false
topic_level = true
predicted_topics_path = "/home/mhillebrand/coref/data/predicted_topics"
data_folder = "data/ecb/mentions"
save_path = "models/pairwise_scorers"
model_path = "models/pairwise_scorers"
model_num = 9
keep_singletons = false
threshold = 44.27621788652271
linkage_type = "average"
Traceback (most recent call last):
  File "predict.py", line 94, in <module>
    data = create_corpus(config, bert_tokenizer, config.split, is_training=False)
  File "/home/mhillebrand/coref/utils.py", line 32, in create_corpus
    with open(config.predicted_topics_path, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/mhillebrand/coref/data/predicted_topics'

Then if I create the data/predicted_topics directory, I get this error:

IsADirectoryError: [Errno 21] Is a directory: '/home/mhillebrand/coref/data/predicted_topics'

$ python run_scorer.py models/pairwise_scorers mixed

Traceback (most recent call last):
  File "run_scorer.py", line 4, in <module>
    from coval.coval.eval import evaluator
ModuleNotFoundError: No module named 'coval.coval.eval'

python get_ecb_data.py --/home/dr/Desktop/corefecb/data /home/dr/Desktop/corefecb 2022-01-03 16:54:15.600300: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1 usage: get_ecb_data.py [-h] [--data_path DATA_PATH] [--output_dir OUTPUT_DIR] get_ecb_data.py: error: unrecognized arguments: --/home/dr/Desktop/corefecb/data /home/dr/Desktop/corefecb why am i getting this error

In run_scorer.py, key_file and sys_file are flipped, i.e. the scorer receives the system predictions instead of the gold annotations (and vice-versa). All recall and precision scores produced by this code section are backwards. :grimacing:

Hi! I'm reproducing this code for a university module regarding reproducibility in NLP. It's going great so far (great job on the reproducible side!) but I'm confused as to how I can obtain the predicted topics (in order to run predict.py with the configuration as "topic:false", it is said I need a path to the predicted topics). Am I meant to get this from another code or are the predicted topics the output of another script on this code? Thank you! :)