Getting good Train Val score but poor Test results

Hi I am using lesson2-image_models.ipynb notebook to train a multilabel image classifier. Below is the result for sz= 256
epoch trn_loss val_loss f2
0 0.142521 0.106025 0.960613
1 0.140658 0.10071 0.96282
2 0.137356 0.100295 0.963254
3 0.136129 0.096858 0.964336
4 0.133358 0.095513 0.965055
5 0.131064 0.095444 0.965335
6 0.129777 0.094232 0.965746

But when I am predicting on test data the result is next to random (~ .54)
Any thoughts why this might be happening?
Edit 1:
I found similar old thread here:
http://forums.fast.ai/t/lesson-3-strange-results-for-planet-dataset/10571
but no solution is available
Thanks in advance!

How are you predicting on the test data set. If you could post the code, will try to help if I could find any differences

Hi,

I am using below code to create the data
def get_data(sz):
tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_top_down, max_zoom=1.05)
return ImageClassifierData.from_csv(PATH, ‘train-jpg’, label_csv, tfms=tfms,
suffix=’.jpg’, val_idxs=val_idxs, test_name=‘test-jpg’)

and I am using below code to find prediction probabilities for test:
multi_preds, y = learn.TTA(is_test = True)
preds = np.mean(multi_preds, 0)

I am using opt_th function from planet.py to obtain threshold and convert float value to integers:

pred1 = (preds > thr).astype(int)

Here my results are deteriorating.(F1 score of .54)

When I am using same process on valid data I am getting good f1 score.(.9+)

First, one pointer: please always format your code, it makes it easier to read

Second: you’re not exponentiating your predictions, which is screwing up your accuracy

Try this instead:

multi_preds, y = learn.TTA(is_test=True)
probs = np.mean(np.exp(multi_preds), axis=0)
accuracy_np(probs,y)

Hi,
Thank you for your reply, using exp creating values greater than 1 and using accuracy_np is throwing error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-31-aee78524c76f> in <module>()
      1 multi_preds, y = learn.TTA()
      2 probs = np.mean(np.exp(multi_preds), axis=0)
----> 3 accuracy_np(probs,y)

~\fastai\courses\dl1\fastai\metrics.py in accuracy_np(preds, targs)
      4 def accuracy_np(preds, targs):
      5     preds = np.argmax(preds, 1)
----> 6     return (preds==targs).mean()
      7 
      8 def accuracy(preds, targs):

AttributeError: 'bool' object has no attribute 'mean'