Multi label classification: List/Show misclassified images in fastai v1

Hello all, I built a model in, which is classifying spare parts for big machines. To account for uncertainty or a ‘no category identified’-class I implemented this as a multi label problem (sigmoid) as recommended by @jeremy here.

To check the accuracy on the test set I first train my model using the real train/val sets. Then I create a new databunch in which I define my labeled test set as the new “fastai val set”, load my trained model and do prediction & accuracy etc. on that test set (now for fastai purposes defined as the val set) as recommended here.

Now I would like to check which images exactly are being misclassified to get an idea of which images the model is having problems with (right now I am at about 85% accuracy for the test set with 110 pictures). @muellerzr shows how this can be done quite easily with the ClassConfusion widget in this thread for softmax. Unfortunately, this doesn’t work with multi label problems and sigmoid. I also get an error if I want to plot the top losses:


IndexError Traceback (most recent call last)
in ()
----> 1 interp.plot_top_losses(9)

4 frames
/usr/local/lib/python3.6/dist-packages/fastai/ in get(self, i)
73 def get(self, i)->Any:
74 “Subclass if you want to customize how to create item i from self.items.”
—> 75 return self.items[i]
76 def repr(self)->str:
77 items = [self[i] for i in range(min(5,len(self.items)))]

IndexError: index 640 is out of bounds for axis 0 with size 110

Is there any way to work around this and either print a list containing the filenames of the misclassified images or show the misclassified pictures for multi label classification?

I already saw in notebook that with v2 we can do this quite easily with

interp = Interpretation.from_learner(learn)

but I would like to stay with v1 for right now, if possible.

Thanks in advance and you help would be greatly appreciated!