Semantic Segmentation Data Loader using Only Data Frame Columns

fastai
1

I am new to fast.ai and have recently transitioned from Keras/Tensorflow. I generally do most of my data analysis in R as opposed to Python, so I apologizes if this is a very basic question or if I have overlooked something here.

I am interested in semantic segmentation for classification of geospatial data. For some experiments I am running, I would like to be able to read image chips from multiple directories/paths. This is because my image chips are derived from different aerial images and I want to be able to control which images are used for training, validation, and testing. I also want to be able to assess how well models trained on a subset of the aerial images generalize to new images.

I would also like to be able to call the path to the associated image mask using a column in a Pandas data frame. I can build a data frame that (1) has one row for each image chip, (2) one column lists the image file path (the full path, not just the name), (3) one column provides the full file path to the associated image mask, and (4) one column differentiates training and validation data using a Boolean. I am able to create a data frame with this structure, but am having trouble building a data loader that can read the image and mask paths from the columns.

In summary, I would like to build a data loader that can:

  1. Read image and chip paths from columns in a Pandas data frame
  2. Allow for reading in files from multiple directories by providing the full file path for each chip and mask in the data frame columns
  3. Differentiates training and validation data based on a Boolean column.

I have struggled with using the .from_df() method since it seems to require a common path. Also, the SegmentationDataLoaders class seems to not be able to call the mask from a column in the data frame.

I would be grateful for any thoughts or advice here.

Thanks.

1 Like
2

are you looking for something like this?

dblock = DataBlock((ImageBlock,MaskBlock),
                   get_x = ColReader('image_col'),#you can additionally give a prefix or suffix,
                   get_y = ColReader('mask_col'),
                   splitter = ColSplitter(col='is_valid')#Name of the column containing boolean values)
                   )

dataloaders = dblock.dataloaders(dataframe)
3

Thanks for this. This seems close now. However, it doesn’t appear to be linking the masks and images. Is this because the random selection is happening separately for the images and masks?

Here is the code I am now using:

dblock = DataBlock((ImageBlock,MaskBlock(codes=codes)),
                   get_x = ColReader('img_path'),
                   get_y = ColReader('msk_path'),
                   splitter = ColSplitter(col='set'),
                   batch_tfms=aug_transforms()
                   )

This is a result from show_batch(). Note that the incorrect mask is assigned to each image.

image

This is the structure of the data frame, where one row represents one image/mask pair:

image

Thanks again for your help with this.

4

This won’t happen.
You can try dblock.summary(dataframe) to see what’s going underneath the creation of dataloaders.

I’m not sure what’s going wrong. Could it be something to do with mapping of codes?

You can also ask for help on discord.

5

The summary report looks good. So, I’m not sure what is going on.

Setting-up type transforms pipelines
Collecting items from img_path
0 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_0.png
1 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_1.png
2 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_10.png
3 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_100.png
4 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_101.png
… …
10669 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_76.png
10670 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_77.png
10671 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_78.png
10672 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_79.png
10673 C:/Maxwell_Data/landcover/output/img/N-34-97-D-c-2-4_9.png

                                                          msk_path

0 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_0_m.png
1 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_1_m.png
2 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_10_m.png
3 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_100_m.png
4 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_101_m.png
… …
10669 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_76_m.png
10670 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_77_m.png
10671 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_78_m.png
10672 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_79_m.png
10673 C:/Maxwell_Data/landcover/output/mask/N-34-97-D-c-2-4_9_m.png

[10674 rows x 2 columns]
Found 10674 items
2 datasets of sizes 8540,2134
Setting up Pipeline: ColReader – {‘cols’: 0, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} -> PILBase.create
Setting up Pipeline: ColReader – {‘cols’: 1, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} -> PILBase.create

Building one sample
Pipeline: ColReader – {‘cols’: 0, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} -> PILBase.create
starting from
img_path C:\Maxwell_Data\landcover\output\img\N-33-139-C-d-2-2_4.png
msk_path C:\Maxwell_Data\landcover\output\mask\N-33-139-C-d-2-2_4_m.png
Name: 7065, dtype: object
applying ColReader – {‘cols’: 0, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} gives
C:\Maxwell_Data\landcover\output\img\N-33-139-C-d-2-2_4.png
applying PILBase.create gives
PILImage mode=RGB size=512x512
Pipeline: ColReader – {‘cols’: 1, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} -> PILBase.create
starting from
img_path C:\Maxwell_Data\landcover\output\img\N-33-139-C-d-2-2_4.png
msk_path C:\Maxwell_Data\landcover\output\mask\N-33-139-C-d-2-2_4_m.png
Name: 7065, dtype: object
applying ColReader – {‘cols’: 1, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} gives
C:\Maxwell_Data\landcover\output\mask\N-33-139-C-d-2-2_4_m.png
applying PILBase.create gives
PILMask mode=L size=512x512

Final sample: (PILImage mode=RGB size=512x512, PILMask mode=L size=512x512)

Collecting items from img_path
0 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_0.png
1 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_1.png
2 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_10.png
3 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_100.png
4 C:\Maxwell_Data\landcover\output\img\M-33-20-D-c-4-2_101.png
… …
10669 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_76.png
10670 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_77.png
10671 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_78.png
10672 C:\Maxwell_Data\landcover\output\img\N-34-97-D-c-2-4_79.png
10673 C:/Maxwell_Data/landcover/output/img/N-34-97-D-c-2-4_9.png

                                                          msk_path

0 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_0_m.png
1 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_1_m.png
2 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_10_m.png
3 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_100_m.png
4 C:\Maxwell_Data\landcover\output\mask\M-33-20-D-c-4-2_101_m.png
… …
10669 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_76_m.png
10670 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_77_m.png
10671 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_78_m.png
10672 C:\Maxwell_Data\landcover\output\mask\N-34-97-D-c-2-4_79_m.png
10673 C:/Maxwell_Data/landcover/output/mask/N-34-97-D-c-2-4_9_m.png

[10674 rows x 2 columns]
Found 10674 items
2 datasets of sizes 8540,2134
Setting up Pipeline: ColReader – {‘cols’: 0, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} -> PILBase.create
Setting up Pipeline: ColReader – {‘cols’: 1, ‘pref’: ‘’, ‘suff’: ‘’, ‘label_delim’: None} -> PILBase.create
Setting up after_item: Pipeline: AddMaskCodes -> ToTensor
Setting up before_batch: Pipeline:
Setting up after_batch: Pipeline: IntToFloatTensor – {‘div’: 255.0, ‘div_mask’: 1}

Building one batch
Applying item_tfms to the first sample:
Pipeline: AddMaskCodes -> ToTensor
starting from
(PILImage mode=RGB size=512x512, PILMask mode=L size=512x512)
applying AddMaskCodes gives
(PILImage mode=RGB size=512x512, PILMask mode=L size=512x512)
applying ToTensor gives
(TensorImage of size 3x512x512, TensorMask of size 512x512)

Adding the next 3 samples

No before_batch transform to apply

Collating items in a batch

Applying batch_tfms to the batch built
Pipeline: IntToFloatTensor – {‘div’: 255.0, ‘div_mask’: 1}
starting from
(TensorImage of size 4x3x512x512, TensorMask of size 4x512x512)
applying IntToFloatTensor – {‘div’: 255.0, ‘div_mask’: 1} gives
(TensorImage of size 4x3x512x512, TensorMask of size 4x512x512)

6

Hi all!

I am also working on semantic segmentation of geospatial data, but I am having the same issue as Aaron. Has anybody found a solution to this? Is the problem perhaps that my files end with _img and _lbl?

Thanks!

Ferris