Reproduce Learner output manually

fastai
1

I struggle to manually reproduce the model output. More precisely:

  • I have a vision model (called learn) trained on MNIST_TINY data
  • learn got the data from a default ImageDataLoaders.from_folder DataLoader

Given a new image I get the predicted class probabilities by calling learn.predict(image)

I like to reproduce these probabilities by hand. I guess my main issue is that I struggle to preprocess the source image the same way as is done by learn. Is there an easy way to extract all preprocessing steps from a trained model and apply those to the source image?

Here is a reproducible example:

!pip install -Uqq fastai
!pip install nbdev

from fastai.vision.all import *
from fastai import metrics, learner
import pathlib
import torch
import torchvision.transforms as transforms

path = untar_data(URLs.MNIST_TINY)

dls = ImageDataLoaders.from_folder(
    path,
    seed = 11,
    train="train",
    valid="valid"
)
dls.show_batch()

learn = vision_learner(dls, models.resnet18, metrics=metrics.error_rate)
learn.fine_tune(1)

# reproduce the probabilities for this guy
myimg = PILImage.create(get_image_files(path)[0])
myimg

# by the model
learn.predict(myimg)

# manually: this fails
# how do I get all transformations?
print(learn.dls.tfms)
print(learn.dls.before_iter)
print(learn.dls.before_batch)

# this is not correct
input_tensor = Pipeline(ToTensor)(myimg).float().cuda()
input_tensor = input_tensor.unsqueeze(0)
input_tensor.shape

from functools import reduce

def manually(model, input_tensor):
  input_tensor = input_tensor.clone()

  for l in model:
    input_tensor = l(input_tensor)

  return nn.Softmax(dim=1)(input_tensor)

model = learn.model
model = model.cuda()

# not the same as in learn.predict(myimg)
manually(model, input_tensor)
2

The following does the magic:

pp = get_image_files(path)[0]

input_tensor = learn.dls.test_dl([pp]).one_batch()[0]
# input_tensor.shape

def manually(model, input_tensor):
  input_tensor = input_tensor.clone()

  for l in model:
    input_tensor = l(input_tensor)

  return nn.Softmax(dim=1)(input_tensor)

model = learn.model
model = model.cuda()
manually(model, input_tensor)