F1 measure not improving with epochs

Part 1 (2018)
2

Hello,

your F1 score function is not working.
Also no threshold is given to get 0/1 outputs for calculating the F1 score.

I used this F1 score function (based on the F2 function in fastai) which delivers comparable results to one in a kaggle competition:

from sklearn.metrics import f1_score

def f1(preds, targs, start=0.17, end=0.24, step=0.01):
    with warnings.catch_warnings():
        warnings.simplefilter("ignore")
        return max([f1_score(targs, (preds>th), average='micro')
                    for th in np.arange(start,end,step)])

Be aware that the function checks for the best threshold and the best threshold is not necessary lying in the defined start and end window.

To get the optimal threshold in your code you can use the ‘opt_th’ function from planet.py which is in the course directory.

Best regards
Michael

3 Likes