I’m looking for a metric that is more representative of the imbalanced image dataset I’m working on. In sklearn the fbeta metric has ‘macro’ and ‘average’ as potential input for the ‘average’ parameter. (link)
With fastai passing nothing to FBeta() results in a notification that the macro will be used. Passing the average instead as a parameter results in nan for the FBeta metric output.
Oops, my mistake. I thought you were trying to pass the fbeta metric function rather then the FBeta class instance.
Let me try tracing the code when I get back to my computer. If I can’t figure it out, we’ll need to ask an expert.
Later…
I don’t see anything wrong with your code, and it returns a reasonable answer when I run it.
There are a couple of places where the code could blow up into NaN, here:
self.metric = (1 + self.beta2) prec rec / (prec * self.beta2 + rec + self.eps)
and
elif avg == “weighted”:
return self.cm.sum(dim=1) / self.cm.sum()
But I hesitate to diagnose without the actual error in front of me, and without knowing the intent of the FBeta code.
You might post a minimal example of the failure. Short of that, fbeta.eps = 1e-2 might localize the overflow and show that precision and recall are both zero.