Aligning prediction inputs and outputs in ULMFiT

For a few evaluation data examples, I would like to print the input, target, and prediction. Using “predict_with_targs” I can get the predictions and targets, but I’m having a lot of trouble getting the input that produced these results. It appears that the outputs do not align to the DataSet or DataLoader (which uses the SortSampler), which I could tell by comparing the target with the “y” variable for these samples.

Here are a few of my attempts:

log_preds, targs = classifier_learner.predict_with_targs()
preds = np.argmax(log_preds, axis=1)

for i in range(10):
#print(’\nContent:’, ’ ‘.join([idx2tok[j] for j in val_dl.dataset.x[i]]))
#print(’\nContent:’, ’ ‘.join([idx2tok[j] for j in[i]]))
print(’\nContent:’, ’ '.join([idx2tok[j] for j in[i]]))

#print('Target:', idx2label[val_dl.dataset.y[i]])
#print('Target:', idx2label[[i]])
print('Target - data.val:', idx2label[[i]])

#this target doesn't match the target above, indicating misalignment
print('Target - pred w targ:', idx2label[targs[i]])
print('Predicted:', idx2label[preds[i]])

I would appreciate any help.

There has been a PR merged today that should help you. get_preds is now overwritten for an RNN_Learner to return the predictions in order.

Excellent, thank you for replying.