Need Help: FastAI to Pytorch Conversion

Deep Learning
1

Hello,

I am working on an image classification problem where there are 13 classes. Training the model went well and my team and I were very happy with how easy it was to train a model and get good metrics quickly. We have been happy with its performance in all of our tests. That being said, it is part of a larger project where all of the other models are native PyTorch. We were asked to convert our model to PyTorch so that everything is consistent for the whole production process. We have followed a few other discussion threads about saving the state_dict and then populating a model. This went well, but the output of our “converted” PyTorch model is different than the output from the FastAI model. Code is below:

from fastai.vision import *
import torch
from skimage.io import imread
from torchvision import transforms
from PIL import Image

model_path = ...

# load fastai model
learn = load_learner(model_path, 'best_model_prod.pkl')

# save as torch
model = learn.model
torch.save(model, model_path + 'fastai-model-arch.pth') #model architecture
torch.save(model.state_dict(), model_path + 'fastai-model-weights.pth') # model weights

device = torch.device("cpu")
model = torch.load(model_path + 'fastai-model-arch.pth') #load model architecutre
model.to(device)
model.eval()
model.load_state_dict(torch.load(model_path + 'fastai-model-weights.pth', map_location=device))

filename = ...
img = imread(filename)
print(img.shape)
print(img.max())
print(img.min())
# (709, 306, 3)
# 255
# 0

# predict using fastai
img = open_image(filename)
probs = learn.predict(img)

print(probs[2])
# tensor([1.4585e-02, 4.0032e-07, 2.8438e-01, 1.4031e-04, 1.2882e-01, 2.5582e-04,
#         5.9365e-06, 2.9052e-05, 4.4165e-04, 3.1288e-04, 2.2289e-03, 2.9547e-03,
#         5.6585e-01])

normalize = transforms.Normalize(mean=imagenet_stats[0], std=imagenet_stats[1])
tfms2 = transforms.Compose([transforms.Resize((224, 224)), transforms.ToTensor(), normalize,])

# predict using pytorch
im = Image.open(filename)
tfImg = tfms2(im)
tfImg = tfImg[None,:,:,:]
softmaxer = torch.nn.Softmax(dim=1)
print(tfImg.shape)
with torch.no_grad():
    raw_out = model(tfImg)
out = softmaxer(raw_out)
print(out[0])

# torch.Size([1, 3, 224, 224])
# tensor([7.4406e-02, 8.0713e-05, 7.4737e-01, 3.3366e-04, 5.8130e-03, 1.0117e-03,
#         1.8871e-04, 1.0999e-03, 2.4550e-02, 2.0531e-02, 3.7975e-03, 3.4103e-02,
#         8.6716e-02])

Any help would be appreciated. I assume that the way that I am pre-processing the data is the culprit, but have not been able to find a good answer to this question.

2

I have looked at the images to see what is happening and here are some results:

original image:

image
image306×709 55.7 KB 

data = DataBunch.load_empty(model_path, fname='data_prod.pkl')

img = open_image(filename)

data.add_test([filename])
learn.data = data

learn.get_preds(DatasetType.Test)
# same as above
# tensor([[1.4585e-02, 4.0032e-07, 2.8438e-01, 1.4031e-04, 1.2882e-01, 2.5582e-04,
#           5.9365e-06, 2.9052e-05, 4.4165e-04, 3.1288e-04, 2.2289e-03, 2.9547e-03,
#           5.6585e-01]])

x,y = next(iter(data.test_dl))
import matplotlib.pyplot as plt
%matplotlib inline
image_np = x.cpu().squeeze_(0).numpy().transpose(1,2,0)
print(image_np.shape)
plt.imshow(image_np)

image

normalize = transforms.Normalize(mean=imagenet_stats[0], std=imagenet_stats[1])
tfms2 = transforms.Compose([transforms.Resize((224, 224)), transforms.ToTensor(), normalize,])

im = Image.open(filename)
tfImg = tfms2(im)
image_np = tfImg.cpu().squeeze_(0).numpy().transpose(1,2,0)
print(image_np.shape)
plt.imshow(image_np)

image

normalize = transforms.Normalize(mean=imagenet_stats[0], std=imagenet_stats[1])
tfms2 = transforms.Compose([transforms.CenterCrop((224, 224)), transforms.ToTensor(), normalize,])

im = Image.open(filename)
tfImg = tfms2(im)
image_np = tfImg.cpu().squeeze_(0).numpy().transpose(1,2,0)
print(image_np.shape)
plt.imshow(image_np)

image

Does anybody know how to crop the same. Looking through the docs and not entirely sure what is happening.