Error when running fastai model on streamlit

Deep Learning
1

Hi, this is my first time posting here, I’m not sure if this is the right place to do so.

but, I need help with an issue I’m facing, I’ve successfully run a model from fast.ai on jupyter notebook and I wanted to showcase it on streamlit, but running the same code, it gave me this error message:
TypeError: unhashable type: ‘PILImage’

the error seems to come from this code block:
inf_dblock = DataBlock(
blocks=(ImageBlock, CategoryBlock),
get_items=lambda x: [Image.open(file)],
item_tfms=Resize(128),
batch_tfms=[Normalize.from_stats(*imagenet_stats)]
)

inf_dls = inf_dblock.dataloaders([], bs=1)

pretrained_model = models.resnet18

model_metrics = [accuracy, partial(top_k_accuracy, k=1),
             partial(top_k_accuracy, k=5)]

learner = load_learner(inf_dls, pretrained_model, model_metrics, model_path)

saved_features = SaveFeatures(learner.model[1][4])

_ = learner.get_preds(inf_dls)

embeddings_list = np.array(saved_features.features).tolist()

return embeddings_list

file is a result of a jpg file upload
file = st.file_uploader(“Upload Image”, type=(“jpg”, “jpeg”, “png”, “bmp”, “webp”))

I tried converting to a hashable type but then the error would be: the dataloader would require an image input, I also tried running the featurizer without dataloader but haven’t found a way yet.

The idea is to use the second last layer from resnet18 model, and use it to obtain a 512-dimensional image embedding with that. and the code ran smoothly on jupyter notebook on my computer and colab.

I’m wondering if anyone knows how to fix this issue. Thank you in advance!

2

It might be the way you are passing get_items. Looking online, it seems that Streamlit uses hashing to manage caching, and PILImage objects are not inherently hashable.

Have you tried updating get_items to Fastai’s get_image_files() function, passing the path as an argument?

Alternatively, you might tweak your lambda function as follows:

path = 'path_to_your_images'
files = get_image_files(path)

inf_dblock = DataBlock(
    blocks=(ImageBlock, CategoryBlock),
    get_items=lambda x: files,  # Return the list of file paths
    item_tfms=Resize(128),
    batch_tfms=[Normalize.from_stats(*imagenet_stats)]
)

If you’ve already tried the above - mind sharing the complete error stack traces?

3

Hi James,

thank you for your help!

I tried the get_image_files option and this is the error message I get:
Error with get_image_files

> TypeError: expected str, bytes or os.PathLike object, not UploadedFile
> Traceback:
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
>     exec(code, module.__dict__)
> File "/Users/joankusuma/Downloads/test-fashion/ctl-dataset/streamlit-app/app.py", line 40, in <module>
>     embeddings_list = img2vec(file, model_path)
>                       ^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 212, in wrapper
>     return cached_func(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 241, in __call__
>     return self._get_or_create_cached_value(args, kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 267, in _get_or_create_cached_value
>     return self._handle_cache_miss(cache, value_key, func_args, func_kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 321, in _handle_cache_miss
>     computed_value = self._info.func(*func_args, **func_kwargs)
>                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Users/joankusuma/Downloads/test-fashion/ctl-dataset/streamlit-app/src/utils.py", line 51, in img2vec
>     files = get_image_files(file)
>             ^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/data/transforms.py", line 61, in get_image_files
>     return get_files(path, extensions=image_extensions, recurse=recurse, folders=folders)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/data/transforms.py", line 32, in get_files
>     path = Path(path)
>            ^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/pathlib.py", line 871, in __new__
>     self = cls._from_parts(args)
>            ^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/pathlib.py", line 509, in _from_parts
>     drv, root, parts = self._parse_args(args)
>                        ^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/pathlib.py", line 493, in _parse_args
>     a = os.fspath(a)
>         ^^^^^^^^^^^^

I also tried returning the hash value of the image like this:

    img = Image.open(file).convert("RGB")
    img = img.resize((128, 128))
    img = io.BytesIO()
    img_hash = hash(img.getvalue())

    inf_dblock = DataBlock(
        blocks=(ImageBlock, CategoryBlock),
        get_items=lambda x: [img_hash],
        item_tfms=Resize(128),
        batch_tfms=[Normalize.from_stats(*imagenet_stats)]
        )

and got the following error message:

> RuntimeError: Expected 3D (unbatched) or 4D (batched) input to conv2d, but got input of size: [1]
> Traceback:
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 534, in _run_script
>     exec(code, module.__dict__)
> File "/Users/joankusuma/Downloads/test-fashion/ctl-dataset/streamlit-app/app.py", line 40, in <module>
>     embeddings_list = img2vec(file, model_path)
>                       ^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 212, in wrapper
>     return cached_func(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 241, in __call__
>     return self._get_or_create_cached_value(args, kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 267, in _get_or_create_cached_value
>     return self._handle_cache_miss(cache, value_key, func_args, func_kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/streamlit/runtime/caching/cache_utils.py", line 321, in _handle_cache_miss
>     computed_value = self._info.func(*func_args, **func_kwargs)
>                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Users/joankusuma/Downloads/test-fashion/ctl-dataset/streamlit-app/src/utils.py", line 74, in img2vec
>     _ = learner.get_preds(dl=inf_dls)
>         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/learner.py", line 308, in get_preds
>     self._do_epoch_validate(dl=dl)
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/learner.py", line 244, in _do_epoch_validate
>     with torch.no_grad(): self._with_events(self.all_batches, 'validate', CancelValidException)
>                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/learner.py", line 199, in _with_events
>     try: self(f'before_{event_type}');  f()
>                                         ^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/learner.py", line 205, in all_batches
>     for o in enumerate(self.dl): self.one_batch(*o)
>                                  ^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/learner.py", line 235, in one_batch
>     self._with_events(self._do_one_batch, 'batch', CancelBatchException)
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/learner.py", line 199, in _with_events
>     try: self(f'before_{event_type}');  f()
>                                         ^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/fastai/learner.py", line 216, in _do_one_batch
>     self.pred = self.model(*self.xb)
>                 ^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
>     return forward_call(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/container.py", line 217, in forward
>     input = module(input)
>             ^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
>     return forward_call(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/container.py", line 217, in forward
>     input = module(input)
>             ^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
>     return forward_call(*args, **kwargs)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 463, in forward
>     return self._conv_forward(input, self.weight, self.bias)
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 459, in _conv_forward
>     return F.conv2d(input, weight, bias, self.stride,
>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

At this moment, I think I’ll need to re-train the model and change the data pre-processing method, so the image will be hashable and match the model’s input expectation, but I’m not sure if that’s the best approach.

4

Interesting. I’m not familiar with Streamlit, but am curious: Have you tried training your model on Colab, Kaggle or even locally (if you have a GPU)? You might then be able to use the resulting pkl file and host the actual predictive app on Streamlit, as is covered in the second lesson.

5

Hi James,

thank you for your help. I did already run the previous model successfully on both colab and my local computer.

I finally was able to successfully run it on streamlit and train a different model with an extra image transformation step so streamlit can easily hash the file, essentially this is the step that I changed:

transformations = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
        ])

def extract_img(image, transformation):
    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    model = FeaturizerModel().to(device)
    model.load_state_dict(torch.load('resnet18-featurizer-3.pt', map_location=device)['model_state_dict'], strict=False)
    model.eval()
    latent_feature = np.zeros((1, 256, 4, 4))
    tensor = transformations(Image.open(image).convert("RGB").resize((128, 128))).to(device)
    latent_feature = model.encoder(tensor.unsqueeze(0)).cpu().detach().numpy()
    del tensor
    gc.collect()
    return latent_feature

Thank you!!

1 Like