Probs.shape v.s.len(data.test_ds.fnames)

Part 1 (2018) Beginner (2018)
1

Hi all,
I am new to fastai. I am running the planet code on different dataset and trying to create a submission file.

I have a question about probs.shape v.s. len(data.test_ds.fnames)

since probs.shape returns # (n_images, n_classes), is the n_images supposed to be the same as what returned by len(data.test_ds.fnames)?

I have probs.shape (7186, 226) and len(data.test_ds.fnames) 39706

I have this question because when I run
df.insert(0, 'id', [o[5:-4] for o in data.test_ds.fnames])

I am getting this:

ValueError: Length of values does not match length of index

I also tried:
submission = pd.DataFrame({'file' : os. listdir(f'{PATH}test'), 'labels' : preds_classes})

and got:

ValueError: arrays must all be same length

the test images in the test folder are 39705.

during the training process (learn.fit), I accidentally deleted some images in the test folder but I put them all back right after. I don’t know if that’s the reason why i only have 7186 images in probs.shape? or the reason why I am getting not the same length error?

Would like to know how I can fix this, thank you.

2

Are probs your predictions on the validation set or the test set?

To get test set prediction, you have to set the is_test parameter to True.

3

It’s on the test set. I tried both:
log_preds_test = learn.predict(is_test = True)
and
prob_preds, y= learn.TTA(is_test=True)