Hi I am on Chapter 4 of the fastbook which discusses the MNIST databases and introduces tensors. Say I have a rank 3 tensor that has a shape of (3,2,2), with the first digit in the shape representing the number of images. So is it accurate to say I have 3 distinct 2x2 matrices?
Yes and no.
Yes, you could refer to this as three 2x2 matrices, and it is technically correct.
However, in PyTorch, a Tensor usually has the shape B x C x H x W where B is the batch size, C represents the number of channels/features, and H and W the height and width of the matrix. So if you had three 50x50 px images in greyscale, the Tensor would have a shape of 3 x 1 x 50 x 50. A color image with RGB channels would then be 3 x 3 x 50 x 50.