When I run the demo.sh in examples directory, this error occurred:

word vectors loaded from ../data/glove.840B.300d.txt
word weights computed from ../auxiliary_data/enwiki_vocab_min200.txt using parameter a=-1.000000
remove the first 0 principal components
Traceback (most recent call last):
  File "sim_sif.py", line 28, in <module>
    parr, sarr = eval.sim_evaluate_all(We, words, weight4ind, sim_algo.weighted_average_sim_rmpc, params)
  File "../src/eval.py", line 64, in sim_evaluate_all
    p,s = sim_getCorrelation(We, words, prefix+i, weight4ind, scoring_function, params)
  File "../src/eval.py", line 13, in sim_getCorrelation
    f = open(f,'r')
IOError: [Errno 2] No such file or directory: '../data/MSRvid2012'

Would you please tell me that:

1. Is this error detrimental to the model training ?
2. Where to download the missing data file ?

Thanks a lot !

I have installed all dependencies, and

cd examples/
./demo.sh

The script has downloaded GloVe glove.840B.300d.txt, but it seems to hangs. No files written in the log folder and no python running on top. Any hint?

Hi,

Thanks for sharing the code! I have some questions about the handling of OOV tokens:

1. How is the sentence embedding computed if any of its words does not have a pretrained vector in GloVe?
2. Say we encounter a word at test time which was not present in the corpus used for estimating word frequencies. How is the weight for this word computed?
3. Do you remove infrequent words when computing the word frequencies from the corpus? I ask because the vocab file name (enwiki_vocab_min200.txt) seems to suggest so. If yes, how are the removed tokens weighted?

Thanks in advance for your time :)

Best, Bhuwan

I would like to run against a nvidia/cuda:8.0-cudnn6-devel-ubuntu16.04 (docker). I have

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.57                 Driver Version: 367.57                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GRID K520           Off  | 0000:00:03.0     Off |                  N/A |
| N/A   38C    P8    17W / 125W |      0MiB /  4036MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

and I could install eventually a BLAS like Intel MKL (that could speed up numpy), etc. Is the current implementation using any gpu low level routine?

Thank you

Getting this error when I try to run sif_embeddings, but I think the issue is with data io. How are the files supplied meant to be used, besides running the demo? I'd like to use SIF for evaluating similarity of sentences I supply. There is no training needed if I were to just use the Glove embeddings correct? What are the neural nets in src used for then?

Thanks!

File "C:\Users\gdev\git\SIF\examples\sif_embedding.py", line 13, in <module>
    (words, We) = data_io.getWordmap(wordfile)
  File "../src\data_io.py", line 12, in getWordmap
    lines = f.readlines()
  File "C:\ProgramData\Anaconda3\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 962: character maps to <undefined>

Hi, thanks for sharing the code.

Would you please point out which part of the code is refer to the "PSL" in your paper ? I only manage to found two weighting functions:

def getWeight(words, word2weight):
...
def getIDFWeight(wordfile, save_file=''):

Hi,

I downloaded and unzipped glove.840B.300d.zip file and than I run python sif_embedding.py. After 30 minutes I receive this error:

Traceback (most recent call last):
  File "sif_embedding.py", line 15, in <module>
    (words, We) = data_io.getWordmap(wordfile)
  File "../src/data_io.py", line 22, in getWordmap
    return (words, np.array(We))
MemoryError

Have you ever seen this error?

Nb: I drop the process after Termial shows this error because the process goes in deadlock/loop (I'm sure it doesn't go forward).

Using cuDNN version 6021 on context None Mapped name None to device cuda: Tesla K80 (1489:00:00.0) ['train.py', '-wordfile', '../data/glove.840B.300d.txt', '-npc', '1', '-dim', '300', '-traindata', '../data/sentiment-train', '-devdata', '../data/sentiment-dev', '-testdata', '../data/sentiment-test', '-layersize', '300', '-nntype', 'proj_sentiment', '-epochs', '10', '-batchsize', '25', '-LW', '1e-06', '-LC', '1e-06', '-memsize', '300', '-learner', 'adam', '-eta', '0.001', '-task', 'sentiment'] Traceback (most recent call last): File "train.py", line 237, in model = proj_model_sentiment(We, params) File "~iclr2017/SIF/src/proj_model_sentiment.py", line 37, in init l_out = lasagne.layers.DenseLayer(l_average, params.layersize, nonlinearity=params.nonlinearity) AttributeError: 'params' object has no attribute 'nonlinearity'

In the algorithm of the paper we can see on line 4 that each v_s should be a column vector of our Matrix X. In the code we can see on line 22, that each datapoint should be a row vector of our Matrix X.

The involved dataset, at line 52343, presents what it seems to be ". . .", but it's not. At this line, the code of the example sif_embedding.py breaks because the split() at line 15 of auxiliary_data/data_io.py splits wrongly the word and its embedding. After a debugging on that line it turned out that the dots of ". . ." are actually dots while the spaces are the code 160 of the extended ASCII table. Probably this file is not encoded in ASCII but in Unicode, however (for practical reasons) the test has been made with ord() so the output is an ASCII code, but the problem doesn't change.

As this code is not maintained anymore I have re implemented SIF heavily reuses this implementation as I needed it for a project. I focused on generating the embedding using SIF. I thank the authors of SIF for such a wonderful paper.

I am sharing my code here.

I am following your framework and extend your work by adding "attention". However, when I review your code, I am confuse that whether you calculated the first principle component with training data or not? If you calculate the first principle component with current dataset (test data), it seems that you introduce some information to create the sentence embeddings for test datasets. In that case, your result may not be accepted.