How to output confidence of classiifcation models

Hello! I am currently using fastaiv1 for single label image classification. I have the model in a fastapi endpoint and would like to return the confidence score along with the label.

def predict_image(data, is_file=True):
img = open_image(data if is_file else BytesIO(data))
prediction = learner.predict(img)[0]
return {‘result’: str(prediction)}

I am using this function to predict with the fastapi endpoint.

Try…

def predict_image(data, is_file=True):
    img = open_image(data if is_file else BytesIO(data))
    label,ndx,probs = learner.predict(img)
    return f"result: {label} probability {probs[ndx]}"

p.s. using three backticks will avoid losing the indent of you code.

2 Likes

ndx is a cool way to abbreviate.

Thanks a ton for this! It worked exactly how I needed it! Just another question. I was reading the learner docs, how did you know you could add the “label,ndx,probs” for this?

I know from:

Looking just now at the learner docs (I presume you mean fastai - Learner, Metrics, Callbacks)…

I actually have trouble parsing this section. It seems more to be documenting the internal representation than beginner usage. I’ve submitted a pull request for a small change that might assist. https://app.reviewnb.com/fastai/fastai/pull/3842/

One last question if you don’t mind. How would I be able to return the label and confidence score in a dictionary?

Its good to preceed questions with a summary of your own research - what you tried that didn’t work. You’ll gain more from doing a little leg work first.
Try this search: How to return a dictionary in python

Hi, I was working on this problem with text classfiication and the ULMFiT langauge models and thought I’d share how I was doing this with get_preds and a test dataset that you want to generate batch predictions on:

# load learner from your pickle file
learn_inf = load_learner('models/model_v_1_0_0.pkl')

# load the text inputs from your dataframe to predict on into a test dataloader
dl = learn.dls.test_dl(df['text']) 

# get_preds ordered=True as default param
preds, _ = learn.get_preds(dl=dl) 

# get probs (values) and arg maxes (indices) of each row
preds_prob_max = preds.max(dim=1)

# get predicted decoded labels
predicted_labels = [learn.dls.vocab[1][p] for p in preds_prob_max.indices]

# get predicted probabilities
predicted_probs = preds_prob_max.values.numpy()

# zip decoded label predictions and probabilities into list of tuples
preds_probs_zip = list(zip(predicted_labels, predicted_probs))

# convert to dataframe
preds_probs_df = pd.DataFrame(preds_probs_zip, columns =['pred_label', 'probability'])
preds_probs_df

Now can work with preds_probs_df to add to concat to your inputs.