I’m working through the fastbook notebooks on GitHub. Overall I’m really enjoying it, but I am having difficulties with the second “Further Research” exercise in Chapter 06:
- Retrain the bear classifier using multi-label classification. See if you can make it work effectively with images that don’t contain any bears, including showing that information in the web application. Try an image with two different kinds of bears. Check whether the accuracy on the single-label dataset is impacted using multi-label classification.
Has anyone been able to successfully complete this exercise? And if so, could they share how they did it?
I’ve been able to convert the original bear classifier to a multi-label classifier; but I can’t get it to work correctly with images that don’t contain bears: It will consistently classify everything as a bear. I’ve tried it with pictures that look nothing like bears (e.g., bicycles), but the classifier still thinks there’s a bear there, and with quite high confidence (often around 80-90%).
This is how I adapted the original
# cf. https://forums.fast.ai/t/how-to-use-bcewithlogitslossflat-in-lesson1-pets-ipynb/59146/5
def multi_l(l): return [l] # amended code bears = DataBlock( blocks=(ImageBlock, MultiCategoryBlock), # changed to MultiCategoryBlock get_items=get_image_files, splitter=RandomSplitter(valid_pct=0.2, seed=42), get_y=Pipeline([RegexLabeller(pat = r'\w+\/([a-z]+)\/\d+\.[jpg|jpeg|JPG]'), multi_l]), item_tfms=Resize(128))
And this is the basic learner I used:
def accuracy_multi(inp, targ, thresh=0.5, sigmoid=True): "Compute accuracy when `inp` and `targ` are the same size." if sigmoid: inp = inp.sigmoid() return ((inp>thresh)==targ.bool()).float().mean() learn = cnn_learner(dls, resnet18, metrics=partial(accuracy_multi, thresh=0.9)) learn.fine_tune(4)
The main things I’ve tried are: changing the threshold of
accuracy_multi (all the way up to
.9999); using deeper versions of ResNet; and freezing the model for multiple epochs, unfreezing it, and then training for one more epoch.
However, nothing seems to work, and my model is unable to recognize the null class - i.e., images without any bears.
Does anyone have any suggestions?