Question about binary classification and discrimination threshold in metrics and on inference

Hi fastai community,
After reading the book and taking the course I started to deep dive into certain areas. At the moment I’m exploring metrics for binary or multi-class classification problems (focused on vision atm). However, I came across couple of subjects such us ROC curve and optimal threshold that I’m struggling to get into it with fastai. Appreciate for any help, and here are the issues I’m facing:

  1. Say, I want to use F1Score() as my metric for my binary classification model (pets vs dogs). Why there is no any threshold parameter to the metric ? I found out that there is a F1ScoreMulti() where threshold can be specified - the doc says it’s for multi-label classification problems - can it be used for binary classification problems ? I think same question is valid for all other similar metrics like Recall, Precision and so on.

  2. Another problem is how to draw the ROC curve and the threshold. I assuming that I can use learner.get_preds() to get the scores in order to draw the curve. Say, I determined the optimal threshold now. How can I use that threshold doing inference ? I can’t see the learner.predict() allows me to do so. Does it mean, that on inference I rather need to do learner.get_preds() but not sure how to threshold those predictions. Can I just threshold those raw predictions ?

Thanks for any suggestions or directions

Thanks !