How to use AUROC with cnn_learner in Fast AI?

Hi All,

I checked the forum and found a few threads talking about how to use AUROC with Fast AI. This is how it was mostly suggested.

acc_02 = partial(accuracy_thresh, thresh=0.2)
f_score = partial(fbeta, thresh=0.2, beta=1)
learn = cnn_learner(data, arch, metrics=[acc_02, f_score], callback_fns=AUROC)

But I got the following error:

Any idea what am I doing wrong here.

Thanks a lot
Kind Regards,

1 Like

You need to instantiate it: pass AUROC().

This is what Fast AI return when I pass the following arguments:

learn = cnn_learner(data, arch, metrics=[acc_02, f_score], callback_fns=AUROC())

Oh, you need to pass it in the metrics, not the callbacks.

Thanks for the response.

I now used the following commands:

learn = cnn_learner(data, arch, metrics=[acc_02, f_score, AUROC()])

And this generates the following output:

1 Like

I know it has been some time, but did you ever get this to work?

No, I tried all the options mentioned to me but all in vain. I didn’t achieve the AUROC with Fast AI.

1 Like

I can confirm that it does work for NLP though. I used exactly this syntax on my text classification data: learn = text_classifier_learner(data_clas, arch=AWD_LSTM, drop_mult=0.5, metrics=[accuracy, AUROC()])


I was using it for multi-label classification in vision task where it always raised an error. I’m not sure what led it into an error if it worked in text classification.


It has been about a month but I will just drop this here. AUROC in the fastai library is restricted to binary classification problems. Use it for multi-classification must be the cause of the error.

1 Like

I am using it for binary classification, but still I’m getting the error. I’d be grateful if anyone helps me with this. Thanks.