How do we use our model against a specific image?

Thank you @quangduong it works.
What is the reason behind it, why do we need to scale it ?
I don’t see that during the training, so why do we have to divide by 255 during the prediction.

Maybe it’s done automatically during training and testing when images are present there in the folders and not when we are explicitly using it against the single image?

As @ecdrid pointed, I took a look into the source code when the images are present in the folders, its pixel values are first scaled into range [0, 1] before further processing (normalized, denormalized, crop, …). I am not sure when we use predict_array() img is scaled or not. But when I compared the transformed img as in this code

im = val_tfms(np.array(Image.open(fname)))

with the one when images are presented in the folder the scales of the two are very different. Plus, you can check the prob

prob = np.argmax(np.exp(pred1))

with the prob when images are presented in folders, again, they are in different magnitude scales. Then, I though something wrong with the scaling and try scale 1/255 as mentioned in previous post and it works.

Yes dividing by 255 fixes this issue of the same class always being returned - thanks.

Edit: Using open_image instead of Image.open will do the divide by 255 as well as converting it to an array so always use that utility function:

im = val_tfms(open_image("/my/image.png"))

Check out source code:

??open_image
3 Likes

Hello ! I am having the same error as @Vikasj (but with AttributeError: ‘PngImageFile’ object has no attribute ‘shape’)

Do you have any idea how to solve it ?


AttributeError Traceback (most recent call last)
in ()
3
4 trn_tfms, val_tfrms = tfms_from_model(arch, sz)
----> 5 im = val_tfrms(Image.open(f’{PATH}valid/dogs/arnaud.jpg’))
6 log_probs = to_np(learn.models.model(V(T(im[None]).cuda())))
7 preds = np.argmax(log_probs, axis=1)

~/fastai/courses/dl1/fastai/transforms.py in call(self, im, y)
501 crop_tfm = crop_fn_lu[crop_type](sz, tfm_y, sz_y)
502 self.tfms = tfms + [crop_tfm, normalizer, ChannelOrder(tfm_y)]
–> 503 def call(self, im, y=None): return compose(im, y, self.tfms)
504 def repr(self): return str(self.tfms)
505

~/fastai/courses/dl1/fastai/transforms.py in compose(im, y, fns)
482 for fn in fns:
483 #pdb.set_trace()
–> 484 im, y =fn(im, y)
485 return im if y is None else (im, y)
486

~/fastai/courses/dl1/fastai/transforms.py in call(self, x, y)
172 def call(self, x, y):
173 self.set_state()
–> 174 x,y = ((self.transform(x),y) if self.tfm_y==TfmType.NO
175 else self.transform(x,y) if self.tfm_y in (TfmType.PIXEL, TfmType.CLASS)
176 else self.transform_coord(x,y))

~/fastai/courses/dl1/fastai/transforms.py in transform(self, x, y)
180
181 def transform(self, x, y=None):
–> 182 x = self.do_transform(x,False)
183 return (x, self.do_transform(y,True)) if y is not None else x
184

~/fastai/courses/dl1/fastai/transforms.py in do_transform(self, x, is_y)
306 def do_transform(self, x, is_y):
307 if is_y: return scale_min(x, self.sz_y, cv2.INTER_NEAREST)
–> 308 else : return scale_min(x, self.sz, cv2.INTER_AREA )
309
310

~/fastai/courses/dl1/fastai/transforms.py in scale_min(im, targ, interpolation)
10 targ (int): target size
11 “”"
—> 12 r,c,*_ = im.shape
13 ratio = targ/min(r,c)
14 sz = (scale_to(c, ratio, targ), scale_to(r, ratio, targ))

AttributeError: ‘PngImageFile’ object has no attribute ‘shape’

This is the recommended approach and will solve most of the problems in this thread.

5 Likes

Hello, I have trained my model and got 90% accuracy,
but somehow when I predict single image it seems to be wrong, I tested more than 20 and all wrong

1 Like

Actually my code isn’t quite right

I’ve been trying to get a single image classified for a few weeks now.
I have yet to figure out what code will make this happen.

Not clear to me what ‘the 2nd return val from tfms_from_model’ is.

In this iteration of my failed attempt I am running…
trn_tfms,val_tfms = tfms_from_model(arch, sz)

im = (open_image(PATH+fn))

im_val = val_tfms(im)

preds = learn.predict_array(im[None])
np.argmax(preds)

and getting the error…
RuntimeError: running_mean should contain 1722 elements not 1024

As far as I understand(from end of lessons 2 I believe) this is the code:
Am I right? @jeremy

But as you see, it didn’t work for me, try run it tell me if it worked for you

im = val_tfrms(Image.open(f’{PATH}valid/dogs/arnaud.jpg’))

Use open_image instead of Image.open. This is the recommended approach and will avoid most problems related to AttributeError: ‘PngImageFile’ object has no attribute ‘shape’.

fn = f’{PATH}valid/dogs/arnaud.jpg’
im = val_tfrms(open_image(fn))
1 Like

Setting learn.precompute = False before you run that prediction will resolved the issue.

trn_tfms, val_tfms = tfms_from_model(arch, sz) # get image transformations
im = val_tfrms(open_image(PATH + fn))
learn.precompute = False # we need to set this since we are passing in an image, not a precomputed activation
preds = learn.predict_array(im[None]) # do prediction
... truncated ...
... truncated ...
2 Likes

Does this classify a totally new image the model hasn’t seen before or do you have to show it something from the validation folder?

I see that this …

learn.precompute = False
i = '/path/to/photo/'
im = val_tfms(open_image(i))
preds = learn.predict_array(im[None])
print (np.argmax(preds))

…returns 1 or 0.
But what can I do to get it to return a value BETWEEN 1 and 0?

Yes, it can do both. I have tested both cases:

  1. Image from the validation set.

  1. Totally new image the model hasn’t seen before.
    Example: https://www.k9rl.com/dog-breeds/toy-dogs/affenpinscher/

3 Likes

It shouldn’t returns 1 or 0 or a value between 1 and 0. Your codes are incorrect. Check my previous reply for the correct codes and how the expected results should look like.

With notebook 1 it classifies dog or cat with 0 or 1. It shows probabilities between 0 and 1.
Not really relevant to my question but I’m trying with my own images with categories of want vs pass. (Stylish images I want to use for a website vs images I don’t want to use).
When I show it a new image it hasn’t seen before it returns a 1 or a 0. I want probilities like it gave on the validation images.
Here it’s giving probabilities (between 1 and 0) that it validated on. It “shouldn’t” you say, maybe because it’s not what the model is supposed to do? But it’s what I want.

(Below) These are new images after the model has been trained. They only return 1 or 0. I want probabilities instead because I know the model isn’t very good yet overall but I know anything below a 0.01 or above a 0.8 is correctly classified. The training it does on my few thousand images has an accuracy of 0.81. Pretty good but I need more images.

So I’m trying to use the model to help me build a bigger, better dataset for it to train on.

I know this is probably too much irrelevant information but, again, I want probilities if that’s possible.

np.argmax() returns the index of an array. Since the array has two elements, the two possibilities for the index are 0 or 1. See: https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.argmax.html

Here is the post with the code that worked best for me (after sifting through this entire thread). probs_single is the array with the probabilities for each of the two classes: http://forums.fast.ai/t/how-do-we-use-our-model-against-a-specific-image/7661/99

I’ve been trying old working examples that no longer work for about an hour now. Where is the most recent code that works to do this?

I have been trying to get single image classification to work on the dogs/cats example from lesson 1, but passing in the images from the validation set is yielding some unexpected results. I have the following code:

PATH = "data/dogscats"
arch = resnet34
sz = 224
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz, aug_tfms=transforms_side_on, max_zoom=1.1))
learn = ConvLearner.pretrained(arch, data, precompute=True)
learn.fit(0.01, 3)

learn.precompute = False
trn_tfms, val_tfms = tfms_from_model(arch, sz)
imgs = np.array([val_tfms(open_image(PATH+'/valid/cats/' + fn)) for fn in os.listdir(PATH+"/valid/cats/")])
log_preds = learn.predict_array(imgs)
preds = np.argmax(log_preds, axis=1)

I can see from the data.classes variable that a 0 indicates a cat and a 1 indicates a dog, but the values in the preds array are almost half and half 1’s and 0’s.

Shouldn’t the values in the preds array be almost entirely 0’s (cats) since these images are from the cats portion of the validation set (aka, the set that the training process used to determine that the model has ~99% accuracy). OR perhaps I have a bug in my code or a fundamental misunderstanding of how the training process works / how the validation set is used. Can anyone set me straight?

Hi, I’m trying to use the saved model against an image, with the following code

#using existing model
# Make transformation model & data
tfmModel = tfms_from_model(resnet34, imSize, aug_tfms = transforms_side_on, max_zoom = 1.1)
data = ImageClassifierData.from_paths(PATH, tfms = tfmModel, bs = batchSize)

myModel = ConvLearner.pretrained(resnet34, data)
myModel.load('180123_resnet34_AllLayer01')

#try to predict the image
trn_tfms, val_tfms = tfms_from_model(resnet34, imSize, aug_tfms = transforms_side_on, max_zoom = 1.1)

url_response = urllib.request.urlopen(url)
im = val_tfms(np.array(Image.open(url_response))/255)
url_log_pred = myModel.predict_array(im[None])

However i encountered the following error:

ValueError: Expected more than 1 value per channel when training, got input size [1, 1024]

However, if I run:
logPred,y = myModel.TTA()

then perform:
url_log_pred = myModel.predict_array(im[None])

then it will be able to run as expected.
Does anyone know why?

If possible I don’t want to run myModel.TTA()