Return Top K accuracy?

fastai
1

Hi, is there a way to return top K accuracy instead of exact match in multi-class classification cases? Thx

1 Like
2

You just have to write the function :wink:

2 Likes
3

I modified this to work for PyTorch versions before 0.4, but the original version (which I’ve lost track of) actually worked for 0.4. At the very least, it should be a good starting point. You’ll just need to set learn.metrics = [accuracy_topk]

# Note - this is for pytorch versions before 0.4
# This is for accuracy in top 3 - you can change the first line to be whatever accuracy you want
def accuracy_topk(output, target, topk=(3,)):
    """Computes the precision@k for the specified values of k"""
    maxk = max(topk)
    batch_size = target.size(0)

    _, pred = output.topk(maxk, 1, True, True)
    pred = pred.t()
    correct = pred.eq(target.view(1, -1).expand_as(pred))

    res = []
    for k in topk:
        correct_k = correct[:k].view(-1).float().sum(0, keepdim=True)
        res.append(correct_k.mul_(100.0 / batch_size))
    return res
8 Likes
5

Thanks a lot.
Indeed a very good starting point.

6

@Tchotchke I’m doing the same thing but for image segmentation. It returns this error:

 correct = pred.eq(target.view(1, -1).expand_as(pred))

  RuntimeError: The expanded size of the tensor (10) must match the existing size (768000) at non- 
 singleton dimension 1.  Target sizes: [1, 10].  Tensor sizes: [1, 768000]

Here is print output:

batch_size = target.size(0)
print(batch_size) # it’s 10
print(pred.size()) #torch.Size([10, 1])
pred = pred.t()
print(pred.size()) # torch.Size([1, 10])

Could you please point me to the right direction>

7

I haven’t worked much with image segmentation, but I think the issue is probably that what you are predicting is different. I’ve been using the top K accuracy for single observation classification - for example, is this movie review positive, negative, or neutral? Or what animal is in this image?

In your case, I think the shape of your prediction might be different and top K accuracy doesn’t really make sense. What’s the metric that is normally used for segmentation?