Hi. I am right now doing Andrew Ng. class (Machine Learning) and am using your answer as a guidance. I've realized something in your Exercise 2 (the regularized exercise one).

You've extracted your data from the ex2data2.txt like this:

data2 = loaddata('data/ex2data2.txt', ',')
y = np.c_[data2[:,2]]
X = data2[:,0:2]

And I believe you didn't inserted the theta_0 parameter to the X variable's first column (supposed to be the value of 1). Aren't it supposed to be so? In the non-regularized logistic regression exercise, you did inserted the theta_0 parameter as like this:

X = np.c_[np.ones((data.shape[0],1)), data[:,0:2]]

Am I missing something here?

Hi Jordi,

First of all, thanks so much for the notebooks. They really help me to follow through the course. I have one question in your notebook 4, nnCostFunction -- where J = ... np.sum((np.log(a3.T)*(y_matrix)+np.log(1-a3).T*(1-y_matrix))).

I think this does matrix multiplication --> giving 10*10 matrix (or n_label * n_label). This gives a matrix, let's name this cost-matrix, Jc. This Jc matrix contains not only how a set of predicted values for one label differs from it's corresponding target (diagonal elements), but also how it is differs from targets of other labels (off-diagonal elements). For example, the multiplication would multiply a column of predicted values np.log(a3.T) of one label (e.g. k) with all columns of targets.

Then the code sums all elements of this matrix. This seems to over-calculate J. Instead of summing all the elements, I think only the diagonal elements are needed.

Please use this picture to accommodate my description, which might be confusing.

Please let me know if I misunderstood the code.

Best regards and thanks again, -Tua

I am referring to your exercise 4 code right now to complete mine. I think you've made a mistake on regularizing the NN backpropagation gradient (if I am wrong, pardon me). This is the equation:

And this is your code:

delta1 = d2.dot(a1) # 25x5000 * 5000x401 = 25x401
delta2 = d3.T.dot(a2) # 10x5000 *5000x26 = 10x26
    
theta1_ = np.c_[np.ones((theta1.shape[0],1)),theta1[:,1:]]
theta2_ = np.c_[np.ones((theta2.shape[0],1)),theta2[:,1:]]
    
theta1_grad = delta1/m + (theta1_*reg)/m
theta2_grad = delta2/m + (theta2_*reg)/m

Shouldn't it be

theta1_ = np.c_[np.zeros((theta1.shape[0],1)),theta1[:,1:]]
theta2_ = np.c_[np.zeros((theta2.shape[0],1)),theta2[:,1:]]

since we do not want to add anything to theta1_grad's and theta2_grad's first column (the bias)?