Can't get attribute 'acc_camvid' - lesson 3

Part 1 (2019)
1

I’ve gone over the image segmentation notebook with the camvid dataset and exported the pickle model using learn.export() to deploy on a flask web app using a hosting service (which means the notebook and flask app are not run on the exact same environments), however when trying to use learn = load_learner(model_path) I get the error:

File ".local/lib/python3.6/site-packages/torch/serialization.py", line 538, in _load result = unpickler.load() AttributeError: Can't get attribute 'acc_camvid' on <module '__main__' (built-in)>

I believe this is because in the notebook, we defined our own metric called acc_camvid with the following function:

name2id = {v:k for k,v in enumerate(codes)}
void_code = name2id['Void']

def acc_camvid(input, target):
    target = target.squeeze(1)
    mask = target != void_code
    return (input.argmax(dim=1)[mask]==target[mask]).float().mean()

I put the function as well as the ‘codes’ array into a file called helpers.py and imported it into my flask app using from helpers import acc_camvid but I still get the error. So I don’t know how to add the attribute. I think I might have to recreate the model somehow but I’m not sure.

Reposted because I got no replies the first time.

2 Likes
2

Rather than exporting the learner and then loading the learner for inference you can just save the weights, and while deployment create an exact same learner as you did for training and then load in the weights. (Here you will require to give data as well, use some dummy data.) I think this might solve your problem.

2 Likes
3

Thank you for the suggestion.

I thought about it and decided I don’t really need the custom metric as was shown in the tutorial so I just omitted it and exported the model as usual with `learn.export()’ and it worked fine.

2 Likes
4

How did you get rid of this error. Can you explain further while loading model in flask?

5

I don’t remember exactly what I did as it was a long time ago but I think I didn’t use the custom metric and used a built-in one instead.

6

How we can do it for unet_learner?

    data_path = Path('/home/data/')
    codes = np.loadtxt(data_path/"codes.txt", dtype=str); codes
    label = data_path/"mask"
    image = data_path/"input"

    get_y_fn = lambda x: label/f'{x.stem}.jpg'

    size = (512,512)#src_size//2
    print(size)
    bs=8

    class SegLabelListCustom(SegmentationLabelList):
        def open(self, fn): return open_mask(fn, div=True)
    
    class SegItemListCustom(ImageList):
         _label_cls = SegLabelListCustom

    src = (SegItemListCustom.from_folder(image)
           .split_by_rand_pct(0.2)
           .label_from_func(get_y_fn, classes=codes) )

    data = (src.transform(get_transforms(), size=size, tfm_y=True )
             .databunch(bs=bs)
             .normalize(imagenet_stats))

    name2id = {v:k for k,v in enumerate(codes)}
    print(name2id)
    # void_code = name2id['Void']

    def acc_camvid(input, target):
        target = target.squeeze(1)
    #     mask = target != void_code
        return (input.argmax(dim=1)==target).float().mean()

    metrics=acc_camvid
    # metrics=accuracy

    wd=1e-2

    learn = unet_learner(data, models.resnet34, metrics=metrics, wd=wd)

    learn.load("./models/stage-2-big");

    img = open_image("test.jpg")

    prediction = learn.predict(img)[0]

    prediction #how to convert it to numpy image?

I have to all this for loading the learner for prediction in the flask application because of load_learner() from .pkl file is not working. Please provide an elegant way of doing this for a single image. After getting the prediction, I am not able to convert it to numpy or OpenCV image format. And If I am trying to save the image, it is saving it a blank image.

Please help me, I am stuck in this!!

7

you may try something like and adjust it to your needs. Please, check lecture notes and blog posts there are many related examples.

images= prediction.cpu()
images-= images.min()
images /= images.max()
images = images.permute(1, 2, 0)
plt.imshow(images);

8

Do you remember which built-in metric you use?

Here is the list of all of them: https://docs.fast.ai/metrics.html

I tried ‘accuracy’ but got an error.

9

Any solution you got for this issue?
Share it with us.

10

All I did was use metrics = accuracy.

11

See reply to mr.ashutoshraj

13

Here’s how I solved the blank image issue. Please keep in mind I did this many months ago and cannot answer specific questions, all I have is my old code to reference.

    img = open_image(img_path) # absolute path

    learn = load_learner(model_path) # absolute path

    prediction = learn.predict(img)

    output = prediction[0]

    output.show(figsize=(5,5), alpha=0.9)

    predictionMask = 'pred_' + filename

    plt.savefig(PATH + predictionMask, transparent=True) # replace PATH
14

This same line throws error. Actually, the problem here is acc_camid was the attribute of main. Therefore at the time of unpickling. It’s trying to find all the attributes. The way round of this error is to create a custom class at the time of pickling and import the same class at the time of inferencing. Or define the metrics=acc_camvid in the main attribute by defining it in the main of python file.

15

Good find, can also just cut and paste the acc_camvid definition into your inference notebook and run the cell before loading the learner

1 Like
16

" Or define the metrics=acc_camvid in the main attribute by defining it in the main of python file"

Could you please elaborate more on this. I am facing the exact same issue in the unet_learner.

17

Same issue while deploying the model in flask. But otherwise, works fine.