I was wondering if anyone had tips on how to integrate the following defined loss function in the fastai framework language model “mean column-wise ROC AUC”. Should we use something like sklearn’s http://scikit-learn.org/stable/modules/generated/sklearn.metrics.roc_auc_score.html#sklearn.metrics.roc_auc_score and then for Y andY_true pass 2 matrices containing all the columns for which the above loss function applies or a different approach is needed? Seems this function supports only 2 vectors, not 2 matrices, but we could do it per column and then avg?

# Column-wise AUC loss function

**radek**#2

The one you link to works with multiple labels. I’ll copy in how I use it with fastai lib when I get to my computer later today.

You need at least one positive example per class for it to work so you might need a large batch size for the validation set.

**radek**#3

This is what I pass in metrics array to fit:

```
def roc_auc(preds, y):
1 return metrics.roc_auc_score(y.data.cpu().numpy(), np.exp(preds.cpu().numpy()))
```

(`metrics`

is just `from sklearn import metrics`

)

**feribg**(Feras) #5

This is the final code I have since it seems that fastai doesn’t automatically one hot encode the Ys:

```
def roc_auc(preds, y):
preds = np.exp(preds) #conv from logs
exp = V(y).data.cpu().numpy() #predicted category ID
#batch X num_classes
bs = preds.shape[0] #batch size
nclass = preds.shape[1] #size to determine length of Y one hot encoding
y = np.zeros((bs, nclass))
y[np.arange(bs), exp] = 1 #one hot encode Ys
return metrics.roc_auc_score(y, preds, average="micro")
```