I checked the forum and found a few threads talking about how to use AUROC with Fast AI. This is how it was mostly suggested.
acc_02 = partial(accuracy_thresh, thresh=0.2)
f_score = partial(fbeta, thresh=0.2, beta=1)
learn = cnn_learner(data, arch, metrics=[acc_02, f_score], callback_fns=AUROC)
But I got the following error:
Any idea what am I doing wrong here.
Thanks a lot
You need to instantiate it: pass
This is what Fast AI return when I pass the following arguments:
learn = cnn_learner(data, arch, metrics=[acc_02, f_score], callback_fns=AUROC())
Oh, you need to pass it in the metrics, not the callbacks.
Thanks for the response.
I now used the following commands:
learn = cnn_learner(data, arch, metrics=[acc_02, f_score, AUROC()])
And this generates the following output:
I know it has been some time, but did you ever get this to work?
No, I tried all the options mentioned to me but all in vain. I didn’t achieve the AUROC with Fast AI.
I can confirm that it does work for NLP though. I used exactly this syntax on my text classification data:
learn = text_classifier_learner(data_clas, arch=AWD_LSTM, drop_mult=0.5, metrics=[accuracy, AUROC()])
I was using it for multi-label classification in vision task where it always raised an error. I’m not sure what led it into an error if it worked in text classification.
It has been about a month but I will just drop this here. AUROC in the fastai library is restricted to binary classification problems. Use it for multi-classification must be the cause of the error.
I am using it for binary classification, but still I’m getting the error. I’d be grateful if anyone helps me with this. Thanks.