Using fast ai version 2 this works on regression problems (Riid competition on Kaggle)
from sklearn.metrics import roc_auc_score
def my_auc(inp, targ):
"Simple wrapper around scikit's roc_auc_score function for regression problems"
inp,targ = flatten_check(inp,targ)
return roc_auc_score(targ.cpu().numpy(), inp.cpu().numpy())
then I used it, like this:
learn = tabular_learner(dls, layers=[200,100], metrics=my_auc)
