Hi fastai community,
After reading the book and taking the course I started to deep dive into certain areas. At the moment I’m exploring metrics for binary or multi-class classification problems (focused on vision atm). However, I came across couple of subjects such us ROC curve and optimal threshold that I’m struggling to get into it with fastai. Appreciate for any help, and here are the issues I’m facing:
-
Say, I want to use
F1Score()
as my metric for my binary classification model (pets vs dogs). Why there is no any threshold parameter to the metric ? I found out that there is aF1ScoreMulti()
where threshold can be specified - the doc says it’s formulti-label
classification problems - can it be used for binary classification problems ? I think same question is valid for all other similar metrics like Recall, Precision and so on. -
Another problem is how to draw the ROC curve and the threshold. I assuming that I can use
learner.get_preds()
to get the scores in order to draw the curve. Say, I determined the optimal threshold now. How can I use that threshold doing inference ? I can’t see thelearner.predict()
allows me to do so. Does it mean, that on inference I rather need to dolearner.get_preds()
but not sure how to threshold those predictions. Can I just threshold those raw predictions ?
Thanks for any suggestions or directions
Thanks !
Marcin