` download dependency parser dataset... (clone from https://github.com/rguthrie3/DeepDependencyParsingProblemSet mkdir: created directory '../dataset/dparser' --2018-03-02 15:08:08-- https://raw.githubusercontent.com/rguthrie3/DeepDependencyParsingProblemSet/master/data/train.txt Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-03-02 15:08:09 ERROR 404: Not Found.

--2018-03-02 15:08:09-- https://raw.githubusercontent.com/rguthrie3/DeepDependencyParsingProblemSet/master/data/vocab.txt Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-03-02 15:08:09 ERROR 404: Not Found.

--2018-03-02 15:08:09-- https://raw.githubusercontent.com/rguthrie3/DeepDependencyParsingProblemSet/master/data/dev.txt Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 151.101.12.133 Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|151.101.12.133|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2018-03-02 15:08:09 ERROR 404: Not Found. `

It seems like @rguthrie3 has deleted the repo... Could you please update a new address for us? Thanks!

Hey,

I am trying to reproduce this notebook, but the loss do not go down as advertised.

[0/5] mean_loss : 1.80
[1/5] mean_loss : 1.64
[2/5] mean_loss : 1.64
[3/5] mean_loss : 1.62
[4/5] mean_loss : 1.76

I am using PyTorch v0.4 on CUDA. My hypothesis is the newer version broke something.

>torch.__version__
'0.4.0a0+a3e9151'

Thanks!

I think this repository is really useful, so I spent many time, and found something.

This pull request is mainly doing 5 things as follows:

1. Make the code fit pep8. I noticed your code fit pep8 somewhere, but somewhere not. To make it more readable to more people as a tutorial and keep coherent, I make all the ten notebooks fit pep8.

2. Add the random seed by random.seed(1024) to make code can be Repeatable. As a tutorial, repeatability is very important. I run the notebook, and everytime get a different result. repeatability can help understand better.

3. Add gpu device_ids support by gpus=[0];torch.cuda.set_device(gpus[0]). pytorch use all gpus as default if torch.cuda.is_available(). Sometimes, some gpus are used fully. And then, the code raises errors.

4. Use dict.get(key) to instead if key in dict.keys() when building word vocabulary. The get method time complexity is O(1), the in method is O(n) due to loop. And always the word vocabulary is huge.

5. Use (3, 4, 5)to replace [3, 4, 5] in notebook 08:CNN-for-Text-classification->CNNClassifier. It's dangerous to use mutable list as the function's default params. It may cause unexpected error. For safty use, I replaced it by tuple, and don't need to change other lines.

I have learned a lot from this elegant project. Thanks a lots! Based on the equation in the Skip-gram-Negative-Sampling algorithm below,

I think the negative example loss calculated by

negative_score = torch.sum(neg_embeds.bmm(center_embeds.transpose(1, 2)).squeeze(2), 1).view(negs.size(0), -1) # BxK -> Bx1 loss = self.logsigmoid(positive_score) + self.logsigmoid(negative_score)

maybe change to negative_score = neg_embeds.bmm(center_embeds.transpose(1, 2)) loss = self.logsigmoid(positive_score) + torch.sum(self.logsigmoid(negative_score), 1) since based on the equation, the negative_socre first goes through a logsigmoid operation, and then sums up.

Hi, thank you for the great repository!

However, I found that for QA type classification, the code includes the sub-type into the training/testing data. Needless to say it's a perfect predictor. So, I am fixing this, plus make a couple of tweaks to make it compatible with the latest Python.

I couldn't get rid of the annoying GenSim warning, but maybe you'll have more luck if you re-run the notebook.

Predictably, after removing the leak the accuracy drops. However, I verified that with ALL the data, it goes up again, at least when you randomly sample 10% from the training set to be test data. Maybe, the numbers will be different if you use the official test set. However, you data download script does not download all the data, so I kept this whole thing as is.

BTW, it takes only a few seconds to train on a MacBook pro (and a couple of minutes on all the data). Not long at all!

Hi, In file 08.CNN-for-Text-Classification.ipynb, where do you pad the input? Is it in [110], line 7: x_p.append(torch.cat([x[i], Variable(LongTensor([word2index['']] * (max_x - x[i].size(1)))).view(1, -1)], 1))? Thanks!

Hi, I have a little question about file 08.CNN-for-Text-Classification.ipynb, [96], line 4: pretrained.append(model[word2index[key]]). word2index[key] means to find key's index, then you should find its pretrained embedding in GoogleNews-vectors-negative300.bin. But the index in this bin file should be different from the index generated from TREC dataset, i.e. model[key's index] may not be this key's (word's) embedding. Thanks!

안녕하세요

좋은 자료 공유해주셔서 정말 감사합니다. 관련 내용을 공부하면서 정말 많은 도움을 받고 있습니다.

Issue에 글을 쓰게된 이유는 다름이 아니라 08.CNN 예제에서 logsoftmax와 cross-entropy의 중복과 관련된 내용을 문의드리기 위함입니다.

CNNClassifier의 output은 모델의 출력값에 log_softmax를 취한 결과를 리턴한다고 되어 있는데요.

후에 모델의 출력 값을 pred라는 변수로 받아서, loss_function(Cross-Entropy)에 input으로 넣어주게 되는데, Pytorch의 Cross-Entropy 함수는 softmax 함수를 통과하기전 raw score의 결과를 input으로 받는다고 알고 있습니다.

따라서, 예제의 코드는 혹시 softmax가 2번 중첩되어 적용되는 것이 아닌지 궁금하여, 문의를 드리게 되었습니다.

감사합니다.

I want to save model for Neural Machine Translation (https://nbviewer.jupyter.org/github/DSKSD/DeepNLP-models-Pytorch/blob/master/notebooks/07.Neural-Machine-Translation-with-Attention.ipynb). Can you help me ?