Hi Denny!
I mentioned in your tutorial about RNNs that I'm facing an error, similar to Edwin's. I actually had exactly the one he's facing, but I already fixed it. It was related to my GPU, which wasn't being correctly enabled.
Now, everything seems to be working fine, I also tested some Theano snippets to see if everything was working ok. However, when I try to execute the sampling code, I'm having the following error (I'll highlight the important parts after it, so you don't have to look directly at this Cthulhu):
Traceback (most recent call last):
File "RNNLM.py", line 345, in <module>
sent = generate_sentence(model)
File "RNNLM.py", line 328, in generate_sentence
next_word_probs = model.forward_propagation(new_sentence)
File "rnn-tutorial-rnnlm/venv/local/lib/python2.7/site-packages/theano/compile/function_module.py", line 606, in __call__
storage_map=self.fn.storage_map)
File "rnn-tutorial-rnnlm/venv/local/lib/python2.7/site-packages/theano/compile/function_module.py", line 595, in __call__
outputs = self.fn()
File "rnn-tutorial-rnnlm/venv/local/lib/python2.7/site-packages/theano/scan_module/scan_op.py", line 672, in rval
r = p(n, [x[0] for x in i], o)
File "rnn-tutorial-rnnlm/venv/local/lib/python2.7/site-packages/theano/scan_module/scan_op.py", line 661, in <lambda>
self, node)
File "scan_perform.pyx", line 356, in theano.scan_module.scan_perform.perform (/root/.theano/compiledir_Linux-3.13--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/scan_perform/mod.cpp:3605)
File "scan_perform.pyx", line 350, in theano.scan_module.scan_perform.perform (/root/.theano/compiledir_Linux-3.13--generic-x86_64-with-Ubuntu-14.04-trusty-x86_64-2.7.6-64/scan_perform/mod.cpp:3537)
ValueError: dimension mismatch in args to gemv (50,50)x(80)->(50)
Apply node that caused the error: GpuGemv{no_inplace}(GpuSubtensor{::, int32}.0, TensorConstant{1.0}, W_copy[cuda], <CudaNdarrayType(float32, vector)>, TensorConstant{1.0})
Inputs types: [CudaNdarrayType(float32, vector), TensorType(float32, scalar), CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, vector), TensorType(float32, scalar)]
HINT: Use another linker then the c linker to have the inputs shapes and strides printed.
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
Apply node that caused the error: forall_inplace,gpu,scan_fn}(Shape_i{0}.0, Subtensor{int64:int64:int8}.0, GpuAlloc{memset_0=True}.0, Shape_i{0}.0, V, U, W)
Inputs types: [TensorType(int64, scalar), TensorType(int32, vector), CudaNdarrayType(float32, matrix), TensorType(int64, scalar), CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix)]
Inputs shapes: [(), (1,), (1, 80), (), (8000, 50), (50, 8000), (50, 50)]
Inputs strides: [(), (4,), (0, 1), (), (50, 1), (8000, 1), (50, 1)]
Inputs values: [array(1), array([0], dtype=int32), 'not shown', array(1), 'not shown', 'not shown', 'not shown']
HINT: Re-running with most Theano optimization disabled could give you a back-trace of when this node was created. This can be done with by setting the Theano flag 'optimizer=fast_compile'. If that does not work, Theano optimizations can be disabled with 'optimizer=None'.
HINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.
The important message is the following:
ValueError: dimension mismatch in args to gemv (50,50)x(80)->(50)
I think this piece of code is referrring to a multiplication between W (of size 50x50) and the hidden layer s_t, but with size 80 (obs: I'm using the same math notation you used, so I don't know if this is mapping directly to the Theano code. I'll check it out).
Do you know what could be causing that?
Sorry for bothering you.
3.8k Feb 11, 2021
9.6k Dec 31, 2022
77 Dec 22, 2022
33 Dec 2, 2022
40 Dec 29, 2022
194 Dec 28, 2022
410 Jan 5, 2023
251 Dec 17, 2022
1.3k Dec 28, 2022
116 Nov 9, 2022
2 Jan 30, 2022
50 Oct 19, 2022
1 Nov 16, 2021
63 May 22, 2021
34 Nov 30, 2022
29 Sep 4, 2022
172 Dec 12, 2022