How to change the threshold of a prediction of multi-label classification

I am trying to create a model based in the content of chapter 6 of the book. I have a multi-label dataset that I’m using to train my model, using, as metrics, an accuracy function such as:

def accuracy_multi1(inp, targ, thresh=0.5, sigmoid=True):
    "Compute accuracy when 'inp' and 'targ' are the same size"
    if sigmoid: inp=inp.sigmoid()
    return ((inp>thresh) == targ.bool()).float().mean()

And my learner is like:

learn = cnn_learner(dls, resnet50, metrics=partial(accuracy_multi1,thresh=0.1))
learn.fine_tune(2,base_lr=3e-2,freeze_epochs=2)

After training my model, I want to predict an image considering the threshold I used a argument, but the method learn.predict('img.jpg') only considers the default thres=0.5. In the following example, my predidction should return True for 'red, ‘shirt’ and ‘shoes’, as their probabilities are above 0.1 (but shoes is below 0.5, so it is not considered as True):

def printclasses(prediction,classes):
    print('Prediction:',prediction[0])
    for i in range(len(classes)):
        print(classes[i],':',bool(prediction[1][i]),'|',float(prediction[2][i]))

printclasses(learn.predict('rose.jpg'),dls.vocab)

Output:

Prediction: ['red', 'shirt']
black : False | 0.007274294272065163
blue : False | 0.0019288889598101377
brown : False | 0.005750810727477074
dress : False | 0.0028723080176860094
green : False | 0.005523672327399254
hoodie : False | 0.1325301229953766
pants : False | 0.009496113285422325
pink : False | 0.0037188702262938023
red : True | 0.9839697480201721
shirt : True | 0.5762518644332886
shoes : False | 0.2752271890640259
shorts : False | 0.0020902694668620825
silver : False | 0.0009014935349114239
skirt : False | 0.0030087409541010857
suit : False | 0.0006510693347081542
white : False | 0.001247694599442184
yellow : False | 0.0015280473744496703

Is there a way to impose the threshold when I’m doing a prediction of a image I’m referencing? Something that would look like:

learn.predict('img.jpg',thresh=0.1)

Hi Breno,

I think that what you want can be achieved by changing the threshold in the loss function. Although you didn’t specify it when you created your Lerner, fastai will select the right loss function for you. In this case, it’d be BCEWithLogitsLossFlat with default value thresh=0.5.

I hence suggest to set another value after the training and before you call the predict method. In your case, I’d add this line: learn.loss_func = BCEWithLogitsLossFlat(thresh=0.1), with the threshold value you prefer.

Hope this is helpful! :wink:

2 Likes