Tabular predict - Test prediction got shuffled

fastai
1

my fastai version = fastai-1.0.54

Hello All

I have been trying to predict new set of data from my trained model
by assigning test data into the learner

# Create new data that I want to predict from new data-frame
src = TabularList.from_df(X_tst[0:4], path=PATH, cat_names=None, cont_names=cont_names)
data = src.split_none().label_empty().databunch(bs=1)

# Assign new data I want to predict into the learner
learn.load('hehehaha')
learn.data.test_dl = data.train_dl

# Get prediction from new test data
preds, _ = learn.get_preds(ds_type=DatasetType.Test)
pred_tst = pd.Series(preds[:,0].tolist())

The problem is when I try to predict let say a bunch of new data

Prediction Results

Input to databunch>>   X_tst[0:1]   X_tst[0:2]   X_tst[0:3]   X_tst[0:4]

Preduction results>>   0.472386     0.472386     0.461183     0.472386
                                    0.462687     0.472386     0.458948
                                                 0.462687     0.462687
                                                              0.461183

I tried adding one row of test data at a time
But after [0:3] the prediction got shuffled all over the place

Any help would be much appreciated

update

After some research I have also tried passing ordered=True into get_preds

preds, _ = learn.get_preds(ds_type=DatasetType.Test, ordered=True)

but then I got this error

TypeError: get_preds() got an unexpected keyword argument 'ordered'

Any help would be much appreciated

3 Likes
2

update

according to

before passing the model to validation, we should run this first

model.eval()

Which I believe is already in the validate function
line 53, basic_train.py

for some reason it works during model training and show correct validation
But when trying to use with new test data, it shuffles my prediction

Any help would be much appreciated

1 Like
3

Apparently, ordered = True works only for text classification models.
I am facing the same issue with vision models, test predictions are not in order.