Anyone can fix this?

Traceback (most recent call last):
  File "11_3_hello_rnn_gpu.py", line 83, in <module>
    outputs = rnn(inputs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "11_3_hello_rnn_gpu.py", line 65, in forward
    out, _ = self.rnn(x, h_0)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 224, in __call__
    result = self.forward(*input, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py", line 162, in forward
    output, hidden = func(input, self.all_weights, hx)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/_functions/rnn.py", line 351, in forward
    return func(input, *fargs, **fkwargs)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/_functions/rnn.py", line 244, in forward
    nexth, output = func(input, hidden, weight)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/_functions/rnn.py", line 84, in forward
    hy, output = inner(input, hidden[l], weight[l])
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/_functions/rnn.py", line 113, in forward
    hidden = inner(input[i], hidden, *weight)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/_functions/rnn.py", line 19, in RNNTanhCell
    hy = F.tanh(F.linear(input, w_ih, b_ih) + F.linear(hidden, w_hh, b_hh))
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py", line 553, in linear
    return torch.addmm(bias, input, weight.t())
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/variable.py", line 924, in addmm
    return cls._blas(Addmm, args, False)
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/variable.py", line 920, in _blas
    return cls.apply(*(tensors + (alpha, beta, inplace)))
  File "/usr/local/lib/python3.6/dist-packages/torch/autograd/_functions/blas.py", line 26, in forward
    matrix1, matrix2, out=output)
TypeError: torch.addmm received an invalid combination of arguments - got (int, torch.cuda.FloatTensor, int, torch.FloatTensor, torch.cuda.FloatTensor, out=torch.cuda.FloatTensor), but expected one of:
 * (torch.cuda.FloatTensor source, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
 * (torch.cuda.FloatTensor source, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
 * (float beta, torch.cuda.FloatTensor source, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
 * (torch.cuda.FloatTensor source, float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
 * (float beta, torch.cuda.FloatTensor source, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
 * (torch.cuda.FloatTensor source, float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
 * (float beta, torch.cuda.FloatTensor source, float alpha, torch.cuda.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
      didn't match because some of the arguments have invalid types: (int, torch.cuda.FloatTensor, int, !torch.FloatTensor!, torch.cuda.FloatTensor, out=torch.cuda.FloatTensor)
 * (float beta, torch.cuda.FloatTensor source, float alpha, torch.cuda.sparse.FloatTensor mat1, torch.cuda.FloatTensor mat2, *, torch.cuda.FloatTensor out)
      didn't match because some of the arguments have invalid types: (int, torch.cuda.FloatTensor, int, !torch.FloatTensor!, torch.cuda.FloatTensor, out=torch.cuda.FloatTensor)

@chsasank, are you potentially interested in this kind of tutorial: https://github.com/hunkim/PyTorchZeroToAll/blob/master/tutorials/PyTorchDataParallel.ipynb

It seems current tutorial (http://pytorch.org/tutorials/beginner/former_torchies/parallelism_tutorial.html) does not have enough information/examples.

Cheers!

Changed all Tensors that were cast to Variables to just Tesnors, as those two classes are now merged. See docs here.

x.data[0] is degraded as well, so I changed those to x.data.item()

Finally, there is a dimension error that comes up when running this. The unsqueeze methods fixes this.

Tested on my system and the file runs with no errors or warnings.

I followed the concept given above , took the data,normalized ,split into train and test data and when training ,it says that it want type long but got float32

Any help would be appreciated

Traceback (most recent call last): File "/home/cxy/chenxiuyi/PyTorchZeroToAll/09_2_softmax_mnist.py", line 94, in test() File "/home/cxy/chenxiuyi/PyTorchZeroToAll/09_2_softmax_mnist.py", line 81, in test test_loss += criterion(output, target, size_average=False).data[0] File "/home/cxy/anaconda2/envs/conda_conda_torch/lib/python2.7/site-packages/torch/nn/modules/module.py", line 224, in call result = self.forward(*input, **kwargs) TypeError: forward() got an unexpected keyword argument 'size_average'

So I use "test_loss += criterion(output,target).data[0]". It's ok! I want to know why you use "size_average=False".

Best regards

I think it would be better to use the length of the list (x_data or y_data) when calculating the average instead of just "3" in line 36 and 38. :D

Dear Professor,

When you asked me to send you a pull request, I checked the code again. And found that there is a little problem in printing infos. And I annotated it.

As for the layer_num and hidden_size, there is a missing assignment in the class. So the self.num_layers is missing. And I've already added it.

Best, Zilin

OS: Ubuntu 18.04 torch: 1.4.0

I got the following error when I ran 12_2_hello_rnn.py:

Traceback (most recent call last):
  File "12_2_hello_rnn.py", line 78, in <module>
	loss += criterion(output, label)
  File "/home/adi/anaconda3/envs/pytorch_dev/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
	result = self.forward(*input, **kwargs)
  File "/home/adi/anaconda3/envs/pytorch_dev/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 916, in forward
	ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/adi/anaconda3/envs/pytorch_dev/lib/python3.6/site-packages/torch/nn/functional.py", line 2021, in cross_entropy
	return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/adi/anaconda3/envs/pytorch_dev/lib/python3.6/site-packages/torch/nn/functional.py", line 1834, in nll_loss
	if input.size(0) != target.size(0):
IndexError: dimension specified as 0 but tensor has no dimensions

I'm guessing the "label" that is being passed while computing the loss is causing the error, since it has 0 dimensions. loss += criterion(output, label)

Which is why I converted it into a 1D LongTensor in the PR. loss += criterion(output, torch.LongTensor([label]))

Another issue popped up after resolving this error:

File "12_2_hello_rnn.py", line 80, in <module>
	print(", epoch: %d, loss: %1.3f" % (epoch + 1, loss.data[0]))
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

Which i resolved by printing the loss directly since it has 0 dimensions (you may use loss.item() as well). The resulting script trained successfully, and the following was the output:

Model(
  (rnn): RNN(5, 5, batch_first=True)
)
predicted string: llllll, epoch: 1, loss: 10.155
predicted string: llllll, epoch: 2, loss: 9.137
predicted string: llllll, epoch: 3, loss: 8.355
predicted string: llllll, epoch: 4, loss: 7.577
predicted string: llllll, epoch: 5, loss: 6.876
predicted string: lhelll, epoch: 6, loss: 6.327
predicted string: ihelll, epoch: 7, loss: 6.014
predicted string: ihelll, epoch: 8, loss: 5.787
predicted string: ihelll, epoch: 9, loss: 5.477
predicted string: ihelll, epoch: 10, loss: 5.274
predicted string: ihelll, epoch: 11, loss: 5.041
predicted string: ihello, epoch: 12, loss: 4.827
predicted string: ihello, epoch: 13, loss: 4.676
predicted string: ihello, epoch: 14, loss: 4.550
predicted string: ihello, epoch: 15, loss: 4.430
predicted string: ihello, epoch: 16, loss: 4.305
predicted string: ihello, epoch: 17, loss: 4.164
predicted string: ihelll, epoch: 18, loss: 4.003
predicted string: ihelll, epoch: 19, loss: 3.860
predicted string: ihelll, epoch: 20, loss: 3.879
predicted string: ihelll, epoch: 21, loss: 3.768
predicted string: ihelll, epoch: 22, loss: 3.642
predicted string: ihelll, epoch: 23, loss: 3.599
predicted string: ihello, epoch: 24, loss: 3.577
predicted string: ihello, epoch: 25, loss: 3.544
predicted string: ihello, epoch: 26, loss: 3.498
predicted string: ihello, epoch: 27, loss: 3.439
predicted string: ihello, epoch: 28, loss: 3.371
predicted string: ihello, epoch: 29, loss: 3.303
predicted string: ihello, epoch: 30, loss: 3.240
predicted string: ihello, epoch: 31, loss: 3.162
predicted string: ihello, epoch: 32, loss: 3.147
predicted string: ihello, epoch: 33, loss: 3.178
predicted string: ihello, epoch: 34, loss: 3.116
predicted string: ihello, epoch: 35, loss: 3.042
predicted string: ihello, epoch: 36, loss: 3.020
predicted string: ihello, epoch: 37, loss: 3.015
predicted string: ihello, epoch: 38, loss: 2.998
predicted string: ihello, epoch: 39, loss: 2.977
predicted string: ihello, epoch: 40, loss: 2.966
predicted string: ihello, epoch: 41, loss: 2.961
predicted string: ihello, epoch: 42, loss: 2.950
predicted string: ihello, epoch: 43, loss: 2.930
predicted string: ihello, epoch: 44, loss: 2.904
predicted string: ihello, epoch: 45, loss: 2.888
predicted string: ihello, epoch: 46, loss: 2.888
predicted string: ihello, epoch: 47, loss: 2.879
predicted string: ihello, epoch: 48, loss: 2.860
predicted string: ihello, epoch: 49, loss: 2.857
predicted string: ihello, epoch: 50, loss: 2.859
predicted string: ihello, epoch: 51, loss: 2.852
predicted string: ihello, epoch: 52, loss: 2.840
predicted string: ihello, epoch: 53, loss: 2.834
predicted string: ihello, epoch: 54, loss: 2.834
predicted string: ihello, epoch: 55, loss: 2.824
predicted string: ihello, epoch: 56, loss: 2.817
predicted string: ihello, epoch: 57, loss: 2.817
predicted string: ihello, epoch: 58, loss: 2.814
predicted string: ihello, epoch: 59, loss: 2.808
predicted string: ihello, epoch: 60, loss: 2.805
predicted string: ihello, epoch: 61, loss: 2.805
predicted string: ihello, epoch: 62, loss: 2.801
predicted string: ihello, epoch: 63, loss: 2.796
predicted string: ihello, epoch: 64, loss: 2.795
predicted string: ihello, epoch: 65, loss: 2.793
predicted string: ihello, epoch: 66, loss: 2.789
predicted string: ihello, epoch: 67, loss: 2.786
predicted string: ihello, epoch: 68, loss: 2.786
predicted string: ihello, epoch: 69, loss: 2.783
predicted string: ihello, epoch: 70, loss: 2.780
predicted string: ihello, epoch: 71, loss: 2.780
predicted string: ihello, epoch: 72, loss: 2.778
predicted string: ihello, epoch: 73, loss: 2.776
predicted string: ihello, epoch: 74, loss: 2.775
predicted string: ihello, epoch: 75, loss: 2.774
predicted string: ihello, epoch: 76, loss: 2.772
predicted string: ihello, epoch: 77, loss: 2.770
predicted string: ihello, epoch: 78, loss: 2.769
predicted string: ihello, epoch: 79, loss: 2.768
predicted string: ihello, epoch: 80, loss: 2.766
predicted string: ihello, epoch: 81, loss: 2.765
predicted string: ihello, epoch: 82, loss: 2.764
predicted string: ihello, epoch: 83, loss: 2.763
predicted string: ihello, epoch: 84, loss: 2.762
predicted string: ihello, epoch: 85, loss: 2.761
predicted string: ihello, epoch: 86, loss: 2.759
predicted string: ihello, epoch: 87, loss: 2.759
predicted string: ihello, epoch: 88, loss: 2.758
predicted string: ihello, epoch: 89, loss: 2.757
predicted string: ihello, epoch: 90, loss: 2.756
predicted string: ihello, epoch: 91, loss: 2.755
predicted string: ihello, epoch: 92, loss: 2.754
predicted string: ihello, epoch: 93, loss: 2.753
predicted string: ihello, epoch: 94, loss: 2.752
predicted string: ihello, epoch: 95, loss: 2.751
predicted string: ihello, epoch: 96, loss: 2.750
predicted string: ihello, epoch: 97, loss: 2.750
predicted string: ihello, epoch: 98, loss: 2.749
predicted string: ihello, epoch: 99, loss: 2.748
predicted string: ihello, epoch: 100, loss: 2.747
Learning finished!

Hi Prof. Kim

When I ran 10_1_cnn_mnist.py on Python 3.5 (Ubuntu 16.04.5 LTS), I met the following error:

10_1_cnn_mnist.py:49: UserWarning: Implicit dimension choice for log_softmax has been deprecated. Change the call to include dim=X as an argument. return F.log_softmax(x) Traceback (most recent call last): File "10_1_cnn_mnist.py", line 92, in train(epoch) File "10_1_cnn_mnist.py", line 69, in train 100. * batch_idx / len(train_loader), loss.data[0])) IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

Look forward to hearing from soon. Many thanks, Jianliang

Hi Dr. Kim,

Why do we need to load data twice into torch tensors with 2 epochs? What the difference of the epoch for dataloader and the epoch for neural network? Sorry, maybe my questions are silly, as I only started learning neural network and PyTorch 2 days ago.

Many thanks, Best wishes, Jianliang

pytorch 1.0.0에서 아래와 같이 에러가 발생해서 출력 메서드를 변경하였습니다.

IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number

Bumps numpy from 1.13.3 to 1.22.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

hi, i watched your video recently and found they are soooooo helpful for me!

will you upload lec 14 video to youtube channel? i really want to learn seq2seq model!

Bumps httplib2 from 0.18.0 to 0.19.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hello I really appreciate the slides, the content is awesome and the way of explaining things is sooooo clear. Could you please share with us exercises solutions to compare our answers?

Error messages were

1. "RuntimeError: dimension specified as 0 but tensor has no dimensions". (line: 24) --> I changed the dimension using '.view'(-1, 1)' to make the label dimension to be 1 instead of "no dimensions"
2. "IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number" (line: 80) --> Since the output of the Cross Entropy Loss here has no dimension, deleted the index term '[0]'.

It was tested with Pytorch 1.1.