I am using wandb, but since the order of validation data is constantly changing, it is difficult to compare different models under the same conditions. Can I somehow fix the order of validation set?
It should always have shuffle=False for the validation, along with drop_last=False. Can you provide how you are building the dataloaders?
Thank you for the reply.
I am just taking the code from the tutorial.
data = ImageDataBunch.from_name_func(path, fnames, labelFunc, ds_tfms=get_transforms(), size=224, bs=64).normalize(imagenet_stats)
And np.random.seed was also fixed.
But every time I get a different result, even with the same model and the same settings. Is this normal?
There may be other seeds that need to be set too. Check here: [Solved] Reproducibility: Where is the randomness coming in?
I’ll check it Thank you!