# Calculating a Specificity Metric

I need to calculate the Sensitivity and Specificity.
According to these:

Sensitivity is the same as Recall, so that one is done.

Recall = tp / tp + fn
Sensitivity = Recall
Precision = tp / tp + fp

All of these are in the fast.ai metrics already:
https://docs.fast.ai/metrics.html

Specificity is defined as:
Specificity = tn / tn + fp

I don’t see it in the metrics.
I also don’t see any way to calculate the TN required for specificity:

So we need tn and fp.
fp is in Precision:

``````def _precision(self):
# Precision = tp / tp + fp
prec = torch.diag(self.cm) / self.cm.sum(dim=0)
if self.average is None: return prec
else:
weights = self._weights(avg=self.average)
return (prec * weights).sum()
``````

So that means that:

# Precision = tp / tp + fp = torch.diag(self.cm) / self.cm.sum(dim=0)

TP = torch.diag(self.cm)
TP + FP = self.cm.sum(dim=0)
So to rearrange:
FP = (TP + FP) - TP = self.cm.sum(dim=0) - torch.diag(self.cm)
I also assume theres a better way since this seems barbaric.

However for TN, I don’t see any examples.
I can see in recall we could do the same trickery to get fn by subtracting tp from self.cm.sum(dim=1)

``````rec = torch.diag(self.cm) / self.cm.sum(dim=1)
``````

So seems that there are two ways to sum the confusion matrix by row or column which result in the fp or the fn of the denominator. Is there a way to get the tn to calculate the specificity, or perhaps a way to get the total results to subtract the other values from it?

I assume the weighting matters for non binary classification scoring but will it change in calculating specificity?

Is it just me or are these literally the worst names for these things in the world. My human brain doesn’t seem to handle binary logic AND its synonyms very well as double negatives and double positives?

Also is there a reason why specificity is not included, it seems popular when discussing medical results, but maybe its fallen out of favour for some reason?

5 Likes

Can anybody help me? I’m going through the same challenge…

Same here! Would appreciate the help.

1 Like

For a temporary solution, I found the following:

interp = ClassificationInterpretation.from_learner(learn)
upp, low = interp.confusion_matrix()
tn, fp = upp, upp
fn, tp = low, low
specificity = tn/(fp + tn)

1 Like