A pytorch implementation of Reading Wikipedia to Answer Open-Domain Questions.

Hi,

Thanks for creating this repo!

When I ran the code with default options and 30 epochs, I got 78.0~78.1 F1 score. Did I miss something? Do I need more training epochs?

Thanks, Tao

The last condition in this function, wherein you return (None, None). Does this condition arise or is it just for avoiding a crash. I am trying to implement the same paper and when I try to get the final labels for my context-question pair, there are many answers that result in ValueError. Is this some flaw in dataset? Thank you.

I ran download.sh and I saw two files in the SQuAD folder: dev-v1.1.json train-v1.1.json

Then I got the error when running: python train.py -e 40 -bs 32

Where can I download the meta.msgpack? Thanks.

在用squad_preprocess.py预处理之后，用load_squad函数load出来的context和question里面UNK太多了，最后的vocab数量40000+，如下图。 但是我把data.msgpack里面存储的id形式的context和question用vocab转化为string形式后，发现UNK太多了，想问一下怎么处理呢？

this argument got renamed for pytorch 1.0: https://pytorch.org/docs/stable/notes/cuda.html can put it into a branch as well if you would like to keep things separate.

say once your model is trained and you export the model for prediction. you want to add all the evidence as database in "id", "txt" format.So multiple users can run the queries on the dataset for Answers . how to add such datasets ? would we require another python script like dataset / document reader.py ?

At first the goal was to add feedback while the script was running, but performance got worse so I gave up. But I fixed some little things for being able to run the file:

1. Add init.py to /drqa so we can import str2bool
2. Add encoding type to load_wv_vocab and build_embedding functions (I don't know if it's my machine configuration but I was getting some UnicodeDecodeErrors)

Does the model map each POS tag and NER tag category to a one-hot encoding? If not, why? It doesn't make sense to me how you can just supply the category ID directly in the embedding.

Hi, I'm new to Machine Learning and I've got a question regarding models predictions. What kind of data do i have to provide? Or even more general question. How do I actually make any predictions using this particular model? Thanks

Hello there!

I was planning to implement the attention-again from this paper Attend It Again on DrQA.

Basically, what Attend It again does is as follows.

This model has two LSTM layers. In the bottom layer of LSTM, we use the traditional attention mechanisms and generate the hidden state of LSTM unit from previous hidden state and current input. Next step, we integrate the hidden state of previous LSTM unit in top layer, current input feature and the current output from the bottom layer of LSTM unit.

My plan was to take the doc_hiddens and the x1_emb and feed these to an Attention similar to qemb_match along with question_hiddens then feed this to a LSTM network similar to doc_rnn. Later take this output and feed into start_attn and end_attn to get the start_scores and end_scores.

Can you please tell, if this will be any good to get the better F1 measure ?

I run the command python interact.py got this error (pt) swapnilbhadade@hitvoice:~/pt/DrQA-1$ python interact.py Traceback (most recent call last): File "interact.py", line 22, in <module> checkpoint = torch.load(args.model_file) File "/home/swapnilbhadade/pt/lib/python3.5/site-packages/torch/serialization.py", line 301, in load f = open(f, 'rb') FileNotFoundError: [Errno 2] No such file or directory: 'models/best_model.pt'

can you tell me how long does it take for the training process to complete?

i am using a google colab notebook. and it has been stuck at epoch 1 since last 20 mins

$ python3 train.py -e 40 -bs 32

02/15/2020 05:17:11 [Program starts. Loading data...] 02/15/2020 05:22:48 {'log_per_updates': 3, 'data_file': 'SQuAD/data.msgpack', 'model_dir': '/Users/balagopalbhallamudi/Desktop/DrQA/models', 'save_last_only': False, 'save_dawn_logs': False, 'seed': 1013, 'cuda': False, 'epochs': 40, 'batch_size': 32, 'resume': '', 'resume_options': False, 'reduce_lr': 0.0, 'optimizer': 'adamax', 'grad_clipping': 10, 'weight_decay': 0, 'learning_rate': 0.1, 'momentum': 0, 'tune_partial': 1000, 'fix_embeddings': False, 'rnn_padding': False, 'question_merge': 'self_attn', 'doc_layers': 3, 'question_layers': 3, 'hidden_size': 128, 'num_features': 4, 'pos': True, 'ner': True, 'use_qemb': True, 'concat_rnn_layers': True, 'dropout_emb': 0.4, 'dropout_rnn': 0.4, 'dropout_rnn_output': True, 'max_len': 15, 'rnn_type': 'lstm', 'pretrained_words': True, 'vocab_size': 91590, 'embedding_dim': 300, 'pos_size': 50, 'ner_size': 19} 02/15/2020 05:22:48 [Data loaded.] 02/15/2020 05:22:48 Epoch 1 02/15/2020 07:07:48 > epoch [ 1] updates[ 2707] train loss[4.38260] remaining[0:00:00]

02/15/2020 07:09:46 dev EM: 53.140964995269634 F1: 64.78947947738538 Traceback (most recent call last): File "train.py", line 377, in main() File "train.py", line 87, in main model.save(model_file, epoch, [em, f1, best_val_score]) File "/Users/balagopalbhallamudi/Desktop/DrQA/drqa/model.py", line 147, in save 'torch_cuda_state': torch.cuda.get_rng_state() File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/cuda/random.py", line 20, in get_rng_state _lazy_init() File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/cuda/init.py", line 196, in _lazy_init _check_driver() File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/cuda/init.py", line 94, in _check_driver raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled (base) Balagopals-MacBook-Pro:DrQA balagopalbhallamudi$ python3 interact.py Traceback (most recent call last): File "interact.py", line 31, in checkpoint = torch.load(args.model_file, map_location=lambda storage, loc: storage) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/serialization.py", line 525, in load with _open_file_like(f, 'rb') as opened_file: File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/serialization.py", line 212, in _open_file_like return _open_file(name_or_buffer, mode) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/torch/serialization.py", line 193, in init super(_open_file, self).init(open(name, mode)) FileNotFoundError: [Errno 2] No such file or directory: 'models/best_model.pt'

hello, i'm the new researcher on machine reading comprehension. when use"python train.py -e 40 -bs 32", the process will stop at “Data loaded”. could you give me the solution about this?

Is it expected that this code can be applied to a Chinese language dataset with only minor changes?

I understand that I will need to provide the following:

• Chinese train/dev data files in the SQuAD format
• GloVe word vectors trained on the Chinese language
• Spacy Chinese language models
• Changes in prepro.py to take care of things such as tokenization, add encoding="utf8" to file read/write statements, etc.

Would very much appreciate any insights if there is any known reasons why this is not supposed to work.

@hitvoice Consider below evidence and questions

{
"evidence":"I am on vacation from July 31st and coming back next month", "question":{
"1":"when he is going on vacation?", "2":"when he is returning back from vacation?", }

Answer will be: "when he is going on vacation?": "July 31st", "when he is returning back from vacation?": "next month",

This is working as expected. But consider the case where I have not provided the return back details and the evidence is just

"evidence":"I am on vacation from July 31st"

And I am getting below answer "when he is going on vacation?": "July 31st", "when he is returning back from vacation?": "July 31st",

And we know that return back date is not July 31st, is there a way to get the score of the prediction and based on some threshold make it invalid or blank?