How to use the ImageClassifierCleaner() function with greyscale images

fastai
1

Hi,

I’m doing a project making an image recognizer which uses labelled greyscale images with associated classes as input. These images can be either JPEG or PIL images, and I’ve tried both formats. The number of colour channels is 1 in either case - and when I call

im = Image.open(filenames[0]); im.mode
the image mode prints as ‘L’ which is how I know they’re greyscale.

I can train a model successfully by:
using the DataBlock API,
creating a dataloaders object from the DataBlock object,
then calling learn.fit_one_cycle(dataloaders)

But after the model is trained I try to use the ImageClassifierCleaner function to show any wrongly labelled data:
cleaner = ImageClassifierCleaner(learn, max_n=2)

Then I get an error which implies that the image was converted from greyscale into RGBA at some point along the way.

I’ll post the stack trace below.

So far I’ve tried building a channel conversion function into the item_tfms pipeline of the DataBlock, but this didn’t seem to work.

I’d like to know if anybody else has had this issue when working with the fastai ImageClassifierCleaner with greyscale images, and what you did to fix it. Or if you have some other piece of knowledge which can help, please let me know.

Thanks!

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File /opt/homebrew/lib/python3.10/site-packages/PIL/JpegImagePlugin.py:639, in _save(im, fp, filename)
    638 try:
--> 639     rawmode = RAWMODE[im.mode]
    640 except KeyError as e:

KeyError: 'RGBA'

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
File /opt/homebrew/lib/python3.10/site-packages/PIL/Image.py:643, in Image._repr_image(self, image_format, **kwargs)
    642 try:
--> 643     self.save(b, image_format, **kwargs)
    644 except Exception as e:

File /opt/homebrew/lib/python3.10/site-packages/PIL/Image.py:2413, in Image.save(self, fp, format, **params)
   2412 try:
-> 2413     save_handler(self, fp, filename)
   2414 except Exception:

File /opt/homebrew/lib/python3.10/site-packages/PIL/JpegImagePlugin.py:642, in _save(im, fp, filename)
    641     msg = f"cannot write mode {im.mode} as JPEG"
--> 642     raise OSError(msg) from e
    644 info = im.encoderinfo

OSError: cannot write mode RGBA as JPEG

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
File /opt/homebrew/lib/python3.10/site-packages/IPython/core/formatters.py:342, in BaseFormatter.__call__(self, obj)
    340     method = get_real_method(obj, self.print_method)
    341     if method is not None:
--> 342         return method()
    343     return None
    344 else:

File /opt/homebrew/lib/python3.10/site-packages/PIL/Image.py:661, in Image._repr_jpeg_(self)
    656 def _repr_jpeg_(self):
    657     """iPython display hook support for JPEG format.
    658 
    659     :returns: JPEG version of the image as bytes
    660     """
--> 661     return self._repr_image("JPEG")

File /opt/homebrew/lib/python3.10/site-packages/PIL/Image.py:646, in Image._repr_image(self, image_format, **kwargs)
    644 except Exception as e:
    645     msg = f"Could not save to {image_format} for display"
--> 646     raise ValueError(msg) from e
    647 return b.getvalue()

ValueError: Could not save to JPEG for display
2

Hi,
you need to convert the images to RGBA

you can try this thread discussion/solution

3

Thanks for your reply.

I am not sure why converting the images to RGBA would be the solution in this case: from what I understand, the images must not have an Alpha channel to be saved as JPEG and displayed in a Jupyter Notebook cell.

So if anything I think I should be trying to ensure that the images are in RGB or L mode, which is what they start off as on disk.

It seems as though somewhere along the way from loading the images to making a Learner, the images might be getting conveted to RGBA.

If the images are converted to RGBA, this is fine for training a model - the fit_one_cycle function works fine and I see the loss curves converging. Also the show_batch() methods work ok and images are displayed in notebook cells.

The issue seems to come later when calling ImageClassifierCleaner(learn)

The ImageClassifierCleaner takes a fastai.learner.Learner as input, then tries to display the images which the Learner points to.

Then I think when the ImageClassifierCleaner tries to display these images, it passes them to jupyter notebook to display, at which point they’re converted into JPEG images and the error occurs because an alpha channel got added somewhere.

Am i understanding this correctly so far?

If so, does anybody know why, or where the images are being converted to RGBA images from greyscale images, or how to change this?

My suspicion is that it’s happening somewhere in the ImageClassifierCleaner when it loads or saves the thumbnails.

Actually now that i’ve dug a little deeper I can see where this is happening in the source code for ImagesCleaner (ImagesCleaner is one of the classes called by ImagesClassifierCleaner)

There’s a function in there called _open_thumb()

Check this out

def _open_thumb(
    fn:Path|str, # A path of an image
    h:int, # Thumbnail Height
    w:int # Thumbnail Width
) -> Image: # `PIL` image to display
    "Opens an image path and returns the thumbnail of the image"
    return Image.open(fn).to_thumb(h, w).convert('RGBA')

So the images are indeed getting conveted to RGBA images.

Update: I have changed the source code so that the conversion made by _open_thumbs in fastai>vision>widgets.py converts the images to RGB instead of RGBA, and now the error doesn’t occur, but now the widget box doesn’t show up either.

I will keep digging around and I’ll post back here once I have a solution.

Thanks!

1 Like
4

Hello Mike,

I am also new and learning on the go, I was also facing the same problem and while searching for solution stumbled upon your post among other solutions. But I guess I learned a bit more from your reply than the other way round :slight_smile: , however there is this another thread if you haven’t seen already.

5

Hi there Aman! I’m glad you learned something from my reply. Good luck and I hope that you figure out the solution. I solved mine and i’m posting below.

6

Ok I figured out what was going on. It wasn’t anything to do with the fastai library per se, but I’ll document it here in case anyone has a similar headache in future.

  1. I didn’t have my terminal paths set up correctly. I use Homebrew and Conda. When I ran export $PATH in the terminal, the homebrew paths came before my conda path, which caused conda and pip packages to be installed in different places on my machine. pip packeges were getting installed to my homebrew path, and conda packages were getting installed to the path for the active conda environment. I checked these from different environments (and different editors) with

which pip,
which python,
which conda

  1. This caused the ipywidgets not to work properly. During the troubleshoot process I tried manually uninstalling, reinstalling and force installing ipython widgets with the following shell commands. This worked, but the underlying cause of this problem was what I described in section 1.
!pip uninstall ipywidgets -y
!pip install ipywidgets

!jupyter nbextension install --py --symlink --sys-prefix widgetsnbextension
!jupyter nbextension enable --py --sys-prefix widgetsnbextension
  1. Because ipywidgets wasn’t working, when I tried to use ImagesClassifierCleaner, the widget didn’t load. The widget can display RGBA images, but the jupyter notebook can not, since it displays images using the JpegImagePlugin. Jpeg doesn’t support RGBA, so I got the error.

So in summary, fastai does support greyscale images, it handles them fine, I just had my programming environment and paths set up wrong. (hint: change the order of things in .bashrc and .bash_profile).

Happy coding!

1 Like
7

Hmmm… not really but I believe it has something to do with libjpeg or libpng installation.
Anyways I have moved on for the time being, as it was not a showstopper for the time being.
I will post if I do crack it…
Good Luck Coding!

8

For folks who run into the same issue, I followed @MikeGallimore’s recommendation here. I ended up simply patching the python file and reloaded the ipykernel. I changed return Image.open(fn).to_thumb(h, w).convert('RGBA') to return Image.open(fn).to_thumb(h, w).convert('RGB') (aka removing the ‘A’ from ‘RGBA’). I’m not very familiar with venvs and site packages and how everything plays together so it’s possible that this breaks something important down the line. I will caution that at the top of the file, it instructs you NOT to modify the file in question.

{{your virtual environment}}/lib/python3.11/site-packages/fastai/vision/wigets.py

2 Likes