I was confused a bit about F1 scores due to some historical code but thought I’d post what worked for me in case it wasn’t clear to others.
I am trying to measure F1 score for the dataset/competition from https://www.kaggle.com/c/quora-insincere-questions-classification. It has a binary classification of 0 or 1.
I’m running 1.0.57 and this is what worked for me:
learn = text_classifier_learner(data_clas, AWD_LSTM, drop_mult=0.5)
learn.metrics = [FBeta(beta=1)]
Using partial(fbeta,beta=1) didn’t work, and neither did Fbeta_binary.