From lesson 1, I want to add in my test set to my data class like so:
arch=resnet34
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz),bs=128,test_name='test')
learn = ConvLearner.pretrained(arch, data, precompute=True)
learn.fit(0.001, 3)
If I run this, with a batch size of 128, using nvidia-smi it uses about 5GB of memory. (I’ve got an 8GB 1070Ti).
Using a batch size of 256 just gives me a cuda out of memory error.
When I don’t include the test_name attribute, I am able to train with a batch size of over 1024 with no memory errors.
As far as I understand it, when training, the model does not see the test data at all. It just uses the train and validation data. So why does the test set matter at all?
Should I be setting up my data object without the test set, train it and then add a test set before predictions?