How do we use our model against a specific image?

I used flask for my little API. I needed to handle a file (Image) upload, and after trying and failing with django, I found flask to be trivial.

Please see the code below.

Some notes. This is for a hackahon, and I have definitely done a hack to predict the images one at a time. I wanted to use 'learner.predict_array() to predict a single image, but the results were never clear to me. What I ended up doing is this. I trained the model in a separate process, similar to what was done in class and in the notebooks. Before I start the flask api, I ensure a single image is in the test directory called ‘placeholder.jpg’. When the api starts up, it creates the pre-trained model and loads the saved trained model.

One endpoint is exposed ‘/upload’ which takes a multi-part form request (typical file upload), extracts the image, saves it in place of the placeholder.jpg image. It then asks the model to predict in test mode. I get the predictions back, do a bit of post-processing, and return a JSON map back to the client. Again, this will work fine for the hackathon, but is def not the ideal way to do single image classification. But then again, I understand that is not necessarily the point of the library. This is just what I am doing to shoehorn that functionality in.

Here is the flask code.

        import os
        import time
        import json
        from flask import Flask, render_template, request
        from fastai.conv_learner import *

        app = Flask(__name__, static_url_path='/static')
        print("Starting flask app...")

        PATH = 'data/shopstyle/'
        f_model = resnet34
        label_csv = f'{PATH}prod_train.csv'
        n = len(list(open(label_csv)))-1
        val_idxs = get_cv_idxs(n)
        sz = 128

        def get_data(sz):
            tfms = tfms_from_model(f_model, sz, aug_tfms=transforms_side_on, max_zoom=1.05)
            return ImageClassifierData.from_csv(PATH, 'train', label_csv, tfms=tfms, suffix='.jpg', val_idxs=val_idxs, test_name='test')

        data = get_data(sz)
        learn = ConvLearner.pretrained(f_model, data)

        def load_model():
            print("Loading model...")
            learn.load(f'{sz}')
            learn.precompute=False

        load_model()

        @app.route('/upload', methods=['POST'])
        def upload_file():
            print("cleaning test dir")
            for root, dirs, files in os.walk(PATH+'/test'):
                for f in files:
                    os.unlink(os.path.join(root, f))
            file = request.files['file']
            f = os.path.join(PATH+"/test", 'placeholder.jpg')
            file.save(f)


            start_time = time.process_time()
            load_model();
            test_preds = learn.predict(is_test=True)
            end_time = time.process_time()
            print("Elapsed time: %.9f" % (end_time-start_time))
            tuples = list(zip(data.classes, test_preds[0]))
            #convert to a float
            tuples = list(map(lambda x: (x[0], float(x[1])), tuples))
            #throw away anything less than 0.1
            result = [item for item in tuples if item[1] > 0.1]
            #return up to 5 categories
            return json.dumps(dict(result[:5])
12 Likes

Thanks a lot…
I will try it today itself and will ping you if I ran into some issues…

Legal issues with the dataset?

1 Like

There is nothing proprietary in this dataset, but since I built it with company data I have to get permission first. Still working on that

Hi @binarypoet This is what worked for me for testing against a single image.

im = val_tfms(open_image(f’{PATH}’+fn)) – Load Image using fastai open_image in dataset.py
learn.precompute=False # We’ll pass in a raw image, not activations
log_preds_single = learn.predict_array(im[None]) – Predict Image
maxP = np.argmax(log_preds_single,axis=1) – Pick the index with highest log probability
probs_single = np.exp(log_preds_single) – If you want the probabilities of the classes
actualclass = data.classes[maxP[0]]

I think the key observation from the above line / code is that the actual classes that you have are one hot encoded and the mapping is in data.classes so you have to look up the actual class from data.classes. Hope this helps

2 Likes

Can you just wrap your code With Three `(top and bottom)

It will make it easier for anyone in future…

Thanks

@ecdrid Wrap your code with Three ?

If the intention is to wrap the code in a function, here it is.

def predictSingleImage(fn):
    im = val_tfms(open_image(f'{PATH}'+fn)) # Load Image using fastai open_image in dataset.py
    learn.precompute=False # We’ll pass in a raw image, not activations
    log_preds_single = learn.predict_array(im[None]) # Predict Image
    maxP = np.argmax(log_preds_single,axis=1) # Pick the index with highest log probability
    probs_single = np.exp(log_preds_single) # If you want the probabilities of the classes
    actualclass = data.classes[maxP[0]] # Look up tactualPT   return actualclass
    return actualclass
4 Likes

I just did that - hope that’s OK @binarypoet

Hello, I am getting this error using this code…

AttributeError Traceback (most recent call last)
in ()
1 trn_tfms, val_tfrms = tfms_from_model(arch, 224)
----> 2 im = val_tfrms(PIL.Image.open(’/home/ubuntu/data/sleevelength/sleeve_length_validation/valid/sleeveless/11459845282725-Vishudh-Black–Red-Printed-Kurta-9521459845282239-1.jpg’))
3


AttributeError: ‘JpegImageFile’ object has no attribute ‘shape’

What am i missing here?

See above here: How do we use our model against a specific image?

There was another user getting same error and the remedy (hopefully) lies in the comment right below the post I linked too.

thanks WG, yes i did find it after commenting :stuck_out_tongue:

Hi,
I’m learning with Cats & Dogs classification.
I’m trying to predict a specific image as follow:

trn_tfms, val_tfms = tfms_from_model(resnet34, sz, aug_tfms = transforms_side_on, max_zoom = 1.1)
im = val_tfms(np.array(Image.open(fname)))

pred1 = learn.predict_array(im[None])
prob = np.argmax(np.exp(pred1))

However, the results of prob is always 0
I found out the pred1 always the following results:
array([[ 0. ,negative_number]], dtype=float32)

Example:
array([[ 0. , -106.45783]], dtype=float32)

It doesn’t matter if the image is dog or cat, the first element of pred1 is always 0.

My learn model has accuracy of 99.3% and the confusion matrix was pretty good.
I’ve checked my learn.data.classes it is ['cats', 'dogs']

Does anybody have similar issue?

I am trying to predict the label for a specific image using code sharing on this thread. I also seeing the different result between predict(isTest=true) and the predict_array() function. I try scaling img/255 before converting it to numpy and now it works. The two results are identical !

1 Like

Try np.array(Image.open(fname))/255. I have a similar issue when working on my dataset. Scaling 1/255 works for me.

2 Likes

Thank you @quangduong! I’ll give it a try as soon as I can.

Thank you @quangduong it works.
What is the reason behind it, why do we need to scale it ?
I don’t see that during the training, so why do we have to divide by 255 during the prediction.

Maybe it’s done automatically during training and testing when images are present there in the folders and not when we are explicitly using it against the single image?

As @ecdrid pointed, I took a look into the source code when the images are present in the folders, its pixel values are first scaled into range [0, 1] before further processing (normalized, denormalized, crop, …). I am not sure when we use predict_array() img is scaled or not. But when I compared the transformed img as in this code

im = val_tfms(np.array(Image.open(fname)))

with the one when images are presented in the folder the scales of the two are very different. Plus, you can check the prob

prob = np.argmax(np.exp(pred1))

with the prob when images are presented in folders, again, they are in different magnitude scales. Then, I though something wrong with the scaling and try scale 1/255 as mentioned in previous post and it works.

Yes dividing by 255 fixes this issue of the same class always being returned - thanks.

Edit: Using open_image instead of Image.open will do the divide by 255 as well as converting it to an array so always use that utility function:

im = val_tfms(open_image("/my/image.png"))

Check out source code:

??open_image
3 Likes

Hello ! I am having the same error as @Vikasj (but with AttributeError: ‘PngImageFile’ object has no attribute ‘shape’)

Do you have any idea how to solve it ?


AttributeError Traceback (most recent call last)
in ()
3
4 trn_tfms, val_tfrms = tfms_from_model(arch, sz)
----> 5 im = val_tfrms(Image.open(f’{PATH}valid/dogs/arnaud.jpg’))
6 log_probs = to_np(learn.models.model(V(T(im[None]).cuda())))
7 preds = np.argmax(log_probs, axis=1)

~/fastai/courses/dl1/fastai/transforms.py in call(self, im, y)
501 crop_tfm = crop_fn_lu[crop_type](sz, tfm_y, sz_y)
502 self.tfms = tfms + [crop_tfm, normalizer, ChannelOrder(tfm_y)]
–> 503 def call(self, im, y=None): return compose(im, y, self.tfms)
504 def repr(self): return str(self.tfms)
505

~/fastai/courses/dl1/fastai/transforms.py in compose(im, y, fns)
482 for fn in fns:
483 #pdb.set_trace()
–> 484 im, y =fn(im, y)
485 return im if y is None else (im, y)
486

~/fastai/courses/dl1/fastai/transforms.py in call(self, x, y)
172 def call(self, x, y):
173 self.set_state()
–> 174 x,y = ((self.transform(x),y) if self.tfm_y==TfmType.NO
175 else self.transform(x,y) if self.tfm_y in (TfmType.PIXEL, TfmType.CLASS)
176 else self.transform_coord(x,y))

~/fastai/courses/dl1/fastai/transforms.py in transform(self, x, y)
180
181 def transform(self, x, y=None):
–> 182 x = self.do_transform(x,False)
183 return (x, self.do_transform(y,True)) if y is not None else x
184

~/fastai/courses/dl1/fastai/transforms.py in do_transform(self, x, is_y)
306 def do_transform(self, x, is_y):
307 if is_y: return scale_min(x, self.sz_y, cv2.INTER_NEAREST)
–> 308 else : return scale_min(x, self.sz, cv2.INTER_AREA )
309
310

~/fastai/courses/dl1/fastai/transforms.py in scale_min(im, targ, interpolation)
10 targ (int): target size
11 “”"
—> 12 r,c,*_ = im.shape
13 ratio = targ/min(r,c)
14 sz = (scale_to(c, ratio, targ), scale_to(r, ratio, targ))

AttributeError: ‘PngImageFile’ object has no attribute ‘shape’

This is the recommended approach and will solve most of the problems in this thread.

5 Likes