Lesson 5: DotProduct

(Rohit Singh) #1

My understanding is that matrix dot product between two 2x2 matrices will result in another 2x2 matrix.

However, the DotProduct example in Lesson 5 results in 2 element vector.
a = T([[1.,2],[3,4]])
b = T([[2.,2],[10,10]])
[torch.FloatTensor of size 2]

The code example uses (a*b).len(1) to compute Dot Product:

class DotProduct(nn.Module):
def forward(self, u, m): return (u*m).sum(1)

Shouldn’t it be
[a b]. [w x] = [aw+by ax+bz]
[c d] [y z] [cw+dy cx+dz]

or in numpy: np.dot(a, b) resulting in:
array([[ 2., 4.],
[30., 40.]])

(Hiromi Suenaga) #2

They are doing element-wise operation, then summing them on the dimension 1.

In other words, a * b will end up looking like [[2, 4], [30, 40]] then .sum(1) will make it [6, 70]

I posted my incomplete notes here https://medium.com/@hiromi_suenaga/deep-learning-2-part-1-lesson-5-dd904506bee8 if you are interested. It makes things easier for me to search when I’m reviewing :slight_smile:

(Rohit Singh) #3

Thanks - this is what I understood. Calling it DotProduct threw me off. Is the naming of that class incorrect?

(Matthijs) #4

A dot product is anything of this nature: a*b + c*d + e*f + ... and in this case that is exactly what happens. It’s not a matrix-matrix multiplication but a dot product between the rows of both matrices.