Calculate metrics on prediction

Deep Learning
1

Hi everyone,

I have an image dataset that has 2 classes, I have split the dataset in two folders: one for training and one for a test set.

Inside each folder there are two sub-folders (one for each class, let’s say: 0 and 1)

I used transfer learning, using a CNN model, Resnet34 and I did it in Google Colab.

import numpy as np
from fastai.vision import *
from fastai.metrics import error_rate

np.random.seed(42)

data = ImageDataBunch.from_folder(train_dir, train=".", valid_pct=0.2, 
       ds_tfms=get_transforms(), size=224, num_workers=4).normalize(imagenet_stats)

learn = create_cnn(data, models.resnet34, metrics=[error_rate,accuracy], wd=1e-1)
learn.fit_one_cycle(4)

learn.unfreeze()
learn.fit_one_cycle(2, max_lr=slice(1e-5,1e-4))
learn.export()

Then what I want to do is to calculate confusion matrix, sensitivity and sensibility on the test set.

learn = load_learner(dataset_dir, test=ImageList.from_folder(test_dir))

preds,y = learn.get_preds(ds_type=DatasetType.Test)

I had some trouble in this way

learn.predict(im)
Instead using learn.predict is predicting correctly the image class from the test set. Basically, predicting in loop each single images I have a better results as predictions. I created a panda.dataframe to store the torch tensors (I am able to do with raw numbers using scikit learn, but in this case I was not able on the tensors), but it starts to be messy and I am not fully convinced is the best approach.

my question is:

how I can calculate confusion matrix, sensitivity and sensibility on the test set?

1 Like