I’m trying to train a model on the Kaggle MNIST dataset, and running into this error after a call to
ValueError: operands could not be broadcast together with shapes (28,28,28) (3,)
All the inputs seemed to be correctly shaped going in, so not sure what the issue is. I’d appreciate any pointers.
Notebook is here: https://github.com/sbc/fastai/blob/master/courses/dl1/my-digits-rec.ipynb
It has got something to do with the data augmentation. That’s why the error only appears with
tfms_from_model the statistics from the imagenet pictures are used and those seem to not work for mnist with the tripled channel. It should work when you use
tfms = tfms_from_stats(None, sz, aug_tfms=transforms_basic).
tfms_from_stats does not work, I forgot to switch precompute off. But it definitely works without augmentation:
data = ImageClassifierData.from_arrays(PATH, trn=(X_train, y_train), val=(X_valid, y_valid), classes=classes, test=np.array(test))
Sorry folks, that was a bit too fast It’s just the shape of the tensor. I think it has to be
(28,28,3) instead of
(3,28,28). When I change:
X = np.stack((df.drop(['label'], axis=1).values.reshape(-1,28,28),)*3, axis=1)
X = np.stack((df.drop(['label'], axis=1).values.reshape(-1,28,28),)*3, axis=3)/255
This works for me. Dividing by 255 is necessary to normalize the data. The same applies to the test data:
test = np.stack((df_test.values.reshape(-1,28,28),)*3, axis=3).astype(np.float32)/255
test = np.stack((df_test.values.reshape(-1,28,28),)*3, axis=1).astype(np.float32)
Thanks, Kai!! That did the trick! I thought I tried 28x28x3 but got a different error.