(with_decoded)Result from learn.model.eval(), I can not get the same good prediction as Fastai's learn class?

fastai
1

I tried to resize the image to 224 and use eval(), which it was used for training. But I still can not get the same result in learn.model(img) and learn.predict. In addition, learn.model(img) way is often wrong without fastai’s learn class.

I guess the problem is the learner class has a with_decoded and I used CrossEntropyLossFlat.
I guess it means the prediction from the model needs to be decoded? If that so, I can even deploy it to ONNX and jit, which I see both of them have the same issue as learning.model.

2

Make sure you’re applying a softmax or argmax to your predictions. fastai’s probabilities are softmaxed (found from the activation in the loss function). Otherwise you’re comparing apples to oranges (raw logits to softmax’d probabilities)

3

from PIL import Image

import torchvision.transforms as transforms

img = Image.open("/content/shio (5).jpg")

resize = transforms.Resize([224, 224])

img = resize(img)

to_tensor = transforms.ToTensor()

img_y = to_tensor(img)

img_y.unsqueeze_(0)

img_y.shape
learn.model.eval()

learn.model.cuda()

this_out = learn.model(img_y.cuda())
F.softmax(this_out).argmax(-1)
//I got output of 4 from about

But I got 5 for learn.predict("/content/shio (5).jpg"), which is correct.
I trained with the tta notebook with loss fuction of CrossEntropyLossFlat. Its act is F.softmax.

5

You haven’t correctly recreated your data from scratch. You’re not normalizing. Also if you’re still using a Learner why not use test_dl? just saw long term goal is ONNX. I have an ONNX port in fastinference if you are interested. It still uses the fastai library however, so if you’re hoping to remove that then you may not be interested

6

Thanks for answering.
My end goal is I can get the same result with f.softmax(learn.eval()(img)) and with learn.predict’s predicted index.

7

Fixing the normalization should do that then :slight_smile:

8 
from PIL import Image

import torchvision.transforms as transforms

img = Image.open("/content/shio (5).jpg")

resize = transforms.Resize([224, 224])

transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))

img = resize(img)

to_tensor = transforms.ToTensor()

img_y = to_tensor(img)

img_y.unsqueeze_(0)

img_y.shape

I just tried this with Fixing the normalization, and only the learn.predict can predict the correct answer.
One day I will reproduce the problem with notebook tta08 or 05 petbreed, and I will post the colab notebook here if anyone is interested.

9

You never use your normalization transform here… (you should do so after your totensor)

10

I would expect it to look something similar to so for that part:

norm = transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
img_y = to_tensor(img)
img_y = norm(img_y)
11

It is still the same.

from PIL import Image

import torchvision.transforms as transforms

img = Image.open("/content/shio (5).jpg")

resize = transforms.Resize([224, 224])

img = resize(img)

to_tensor_and_norm = transforms.Compose([

transforms.ToTensor(),

transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)),

])

img_y = to_tensor(img)

img_y.unsqueeze_(0)

img_y.shape
learn.model.eval()

learn.model.cuda()

this_out = learn.model(img_y.cuda())

F.softmax(this_out).argmax(-1)   //outputs 4, but correct index should be 5 and learn.predict got it.
12

You’re not calling your to_tensor_and_norm

1 Like
13

Yes, you are correct. I got index of 5. Thanks for taking time. :laughing:

1 Like
14

Glad we finally got some success!!