Is it because it haven’t see any other images?
Will it work if I create a new category called others and use images from a dataset like ATLAS?
(Anyway, going to try this soon)
Not sure why it’s giving high probs for cat/dog. Perhaps you can try hooking in GradCAM to see which part of your test image (non-dog/cat) is giving high activations for dog/cat?
As for the predictions, I think you will need to do some trial-and-error to decide on a good ‘threshold’ probability to use. The threshold should also form part of the accuracy metric you look at, to inform your training. @muellerzr’s notebook here shows an example of the threshold accuracy metric.
Although it is giving you 0.7458 it is still nowhere near as high as your values for the dog and cat in your notebook and I would guess nowhere near as high as if you used a CatergoryBlock with a softmax. Have you tried looking at the probabilities you get when classifying your validation set to get an appropriate threshold?
Change the loss function to bcewithlogits, this has been discussed before in the forums also in the previous version of the course. Zack also had posted a notebook to showcase this.
Using MultiCategoryBlock should do this automatically. It definitely used to and from the output I think it still does (if not then the probabilities (0.7006 and 0.7458) would add to 1).
I did some experiments on this and I think threshold can make a big difference.
But I think the notebook posted by @utkb is somewhat incorrect because it was only used in accuracy. Even that, I think threshold of 0.1 is not correct.
I think we need to do use the threshold when predicting. But I couldn’t find a way to do it. So, I did this.
Then I tried to load the pascal dataset and try to predict on that.
I couldn’t find a way to do a batch prediction and so I read some source code and did something
See, there’s something wrong here. Could you help me to dig into this?
IIRC there’s a with_decoded parameter to predict. This should apply the LossLogits decode which should help. But I don’t believe I found you need to. It should stick to the thresh you set during training (as it’s on the metrics side not the loss function side)