I created a learner for testing performance on new data.
I load there the weights of a training with very good accuracy on train and valid set - error rate of 0.027 on train and 0.0045 on valid)
When I run validate on the new data, I get an error rate of 3.098.
How can it even be more than 1?
I created 2 test sets (one for each class) to inspect the predictions. And on them, I see that the outputs of the network are excellent… so this error rate of 3 is very confusing. Can someone help?
real_data_idan_val = ImageDataBunch.from_folder(path, ds_tfms=get_transforms(do_flip=False), bs=bs,size=sz,train='TRAININGSET',valid='REAL_IDAN_VALID',resize_method=ResizeMethod.SQUISH) `learn_for_real_idan_val = create_cnn(real_data_idan_val, models.resnet50, metrics=error_rate)` learn_for_real_idan_val.load('fit_one_cycle_3epochs_00045error'); learn_for_real_idan_val.validate(real_data_idan_val.valid_dl) Out:[3.098189, tensor(0.3611)]