Learner.validate() gives me wrong metrics scores

My fastai version is 1.0.51.

I’m working on a multi-label classification problem and use fbeta as metrics.
After training a few epochs, I use the code below to check fbeta score:
learner.validate(learner.data.valid_dl)
and it gave me:
[0.010854037, tensor(0.3351)], here fbeta score is 0.3351.

And then, I tried the code below to get fbeta score:
fbeta(*learner.get_preds(ds_type=DatasetType.Valid))
however, it gave me:
tensor(0.0145), here fbeta score is 0.0145.

Why is that?



2 Likes

Hi
I observed the same thing while doing predictions on a dataset.

In my case, I found that in .validate(), the parameter sigmoid to fbeta function was being passed as False

So, maybe try
fbeta(*learner.get_preds(ds_type=DatasetType.Valid), sigmoid=False)