I just tried out the example calls and got an error:

python -u score_sentence_pairs.py --sentence-pair-file paraphrase-at-scale/example-sentences-pairs.txt --load-file paraphrase-at-scale/model.para.lc.100.pt  --sp-model paraphrase-at-scale/paranmt.model --gpu 0

Traceback (most recent call last):
  File "score_sentence_pairs.py", line 88, in <module>
    model, _ = load_model(None, args)
  File "/content/paraphrastic-representations-at-scale/models.py", line 35, in load_model
    model = Averaging(data, model_args, vocab, vocab_fr)
  File "/content/paraphrastic-representations-at-scale/models.py", line 205, in __init__
    super(Averaging, self).__init__(data, args, vocab, vocab_fr)
  File "/content/paraphrastic-representations-at-scale/models.py", line 52, in __init__
    self.report_interval = args.report_interval
AttributeError: 'Namespace' object has no attribute 'report_interval'

https://github.com/jwieting/paraphrastic-representations-at-scale/blob/1e49a4a7de1619a189e63cce7cd4f3138b53e7a8/models.py#L51-L52

Replacing args.report_interval with args.save_interval let's me run the example but is no real solution...

It seems like the code was maybe later changed but the saved models do not contain the changes:
model = torch.load(args.load_file) → model_args = model['args'] → model_args.report_interval ?

HI @jwieting, I am confused how this particular line implements the loss mentioned in the paper? https://github.com/jwieting/paraphrastic-representations-at-scale/blob/20c66dd01002da5eb9e239e613afc598ecc71b06/models.py#L139

What are g1,g2,p1,p2 ?

First thank you for your work "Paraphrastic Representations at Scale"😝. In this paper you said you released trained models for English, Arabic, German, French, Spanish, Russian, Turkish, and Chinese in abstract. Where can I download chinese model? If you can provide, thank you very much.

First of all, thank you for your contribution to the STS task, we're very excited to get our hands on the models you provided! 😊

Problem

According to the README, when we want to use the pre-trained ParaNMT model model.para.lc.100.pt for scoring sentence pairs (we tested also fine-tuning), it should be used with the sentencepiece model paranmt.model.

# README.md

python -u score_sentence_pairs.py \
  --sentence-pair-file paraphrase-at-scale/example-sentences-pairs.txt
  --load-file paraphrase-at-scale/model.para.lc.100.pt \
  --sp-model paraphrase-at-scale/paranmt.model \
   --gpu 0

However, we observe a potential mismatch between the model.para.lc.100.pt embedding layer size and the size of vocabulary of the paranmt.model sentencepiece model.

• model.para.lc.100.pt has an embedding layer of shape Embedding(82983, 1024).
• paranmt.model sentencepiece model has a vocab size of 50000.

We analyzed the situation further by printing various tokens from the paranmt.model sentencepiece model and they are identical with tokens in paranmt.vocab. However model.para.lc.100.pt was most probably trained using paranmt.sim-low=0.4-sim-high=1.0-ovl=0.7.final.vocab, whose size is exactly 82982 tokens (don't know why the shift of 1 is there 😄 ). We thus believe that a different sentencepiece model (some that has 82983 tokens) should be used with the model.para.lc.100.pt in order to get correct results.

# score_sentence_pairs.py

model, _ = load_model(None, args)
print(model.embedding)  # Embedding(82983, 1024)
print(model.sp.vocab_size())  # 50000

# All of the following tokens agree with the order in the `paranmt.vocab`
print(model.sp.id_to_piece(0))  # <unk>
print(model.sp.id_to_piece(4))  # _the
print(model.sp.id_to_piece(10))  # _i
print(model.sp.piece_to_id('▁i'))  # 10

Possible solutions

We see two possible solutions:

1. Publish the sentencepiece model that was used for training of the model.para.lc.100.pt model.
2. Publish a model trained on ParaNMT that used the paranmt.model sentencepiece model during training. Safety check is that the model should have embedding layer of size 50000.

If we missed something, could you please explain how the models were meant to be used? If we're right, would it please be possible to share the remaining resources?

Hi, I noticed that not all the args the user specified for the main.py are fed into the load_model function and some of the args will still be the same as their pre-training setting. Maybe the model_args in line 35 and 37 from the main.py file should be changed into args?

The SentencePiece trainer in the paranmt preprocessing script preprocess_data.py fails for me on a machine with 16GB of RAM. The memory requirement appears to be around 17GB. It would be helpful make note of this in a readme file as the error message wasn't very informative.

do_all.sh: line 32: 23983 Killed                  python preprocess_data.py --lower-case 1 --paranmt-file scratch/paranmt.sim-low=$1-sim-high=$2-ovl=$3.txt --name "sim-low=$1-sim-high=$2-ovl=$3"