Is there any way to fix the order of validation data? (not to be shuffled)

fastai
1

I am using wandb, but since the order of validation data is constantly changing, it is difficult to compare different models under the same conditions. Can I somehow fix the order of validation set?

2

It should always have shuffle=False for the validation, along with drop_last=False. Can you provide how you are building the dataloaders?

3

Thank you for the reply.

I am just taking the code from the tutorial.

i.e.
data = ImageDataBunch.from_name_func(path, fnames, labelFunc, ds_tfms=get_transforms(), size=224, bs=64).normalize(imagenet_stats)

And np.random.seed was also fixed.

But every time I get a different result, even with the same model and the same settings. Is this normal?

4

There may be other seeds that need to be set too. Check here: [Solved] Reproducibility: Where is the randomness coming in?

1 Like
5

I’ll check it Thank you!