Hello,
Using the script below, I get different results when I use the same data on my testset and validationset. Am I missing something? Thanks in advance!
X_test = [Path("./img1.jpg"), Path("./img2.jpg")]
datablock = DataBlock(blocks=(ImageBlock, MaskBlock(codes)),
get_items=get_image_files,
splitter=FuncSplitter(lambda o: o in X_test),
get_y=get_y_fn,
item_tfms=item_tfms,
batch_tfms=[*aug_tfms, Normalize.from_stats(*imagenet_stats)])
dataloaders = datablock.dataloaders(path_img, path=path, bs=bs)
#### test dataloader ####
test_dl = dataloaders.test_dl(X_test, with_labels=True)
test_dl.vocab = codes
#### check if model overfits ####
print(learner.validate(dl=dataloaders.train))
#### validate validset ####
print(learner.validate(dl=dataloaders.valid))
#### validate testset ####
print(learner.validate(dl=test_dl))
When using this code, I get the following output:
dataloaders.train: [1.5848780870437622, 0.22719594836235046, 0.0376924449941295, 0.017072551355944307]
dataloaders.valid: [1.5183881521224976, 0.20538781583309174, 0.036572121452386336, 0.01625333443666005]
test_dl: [1.6501643657684326, 0.07489459961652756, 0.015047324393568559, 0.002782088336188925]
Test_dl and dataloaders.valid should give the same result