Do fastai data loaders require images and labels to be files on disk?

Looking for an answer to this question if anyone has a moment: Image Segmentation on COCO dataset - summary, questions and suggestions - #5 by austinmw

“Any chance you’ve figured out how to use the json data directly instead of needing to create the intermediary mask images?”

I have mask annotations compressed in COCO format in RLE format (sparse arrays). They don’t need to be saved as png files because the sparse representation for these masks fits in json format and takes up less space than many png files. I haven’t found examples or documentation indicating that fastai data loaders accept a get_y function that can return the array data directly, but wanted to see if there was some way to create a fastai data loader that does support a get_y function that returns the array data rather than a filename. Any help or pointers are very appreciated!

Some mroe detail:
I’ve looked over this tutorial mainly, it seems like this example shows that MaskBlock only expects filenames for the get_y function: Data block tutorial | fastai
I’d like to convert the instance segmentation annotations in my COCO dataset to semantic segmentation format in memory without intermediate saving of label files.

Have you tried writing a custom get_y function that returns a np ndarray from your decoded rle json files? The MaskBlock should take the results of the get_y function and pass them to the PILMask.create function which accepts a number of types including a numpy ndarray.

It’s been a while since I worked with RLE encoding, but from what I remember doing the RLE decoding was slower than opening a png file.

Hopefully this helps!

1 Like

Ah, thanks for this! I didn’t think to check the PILMask.create function, I just assumed it only took paths. I’ll give this a go, thanks a bunch for the suggestion.