'Labels were not included in the Training Set'

Part 1 (2019)
1

Hi all,

I’m implementing my own multi-label classifier. Trying to identify raw material of stone tools (i.e quartzite, andesite etc.)

My issue is that when training the basic model it occasionally throws the error '“Labels ‘x’ were not included in the training dataset” where x is a raw material. Is there a simple way to ensure that all the labels are included in the training dataset?

Cheers.

2

Is there more than one sample of each potential class? If so, you can use something like StratifiedKFold (or a similar strategy – check out the sklearn model selection line below) to ensure that each sample is represented in both the training and test set. You’ll need to use the IndexSplitter instead of RandomSplitter or whatever was being used before – you will just need to pass the validation index values from the SKFold output

If you do not have at least one of each sample, you can make this the case by oversampling the under-represented rows.

Here’s the IndexSplitter in fastai: Helper functions for processing data and basic transforms | fastai

Here’s the SKFold docs: sklearn.model_selection.StratifiedKFold — scikit-learn 0.24.2 documentation

Here’s the overall model selection page in sklearn: API Reference — scikit-learn 0.24.2 documentation

1 Like
3

Hi Kieran

You can try increasing the random splitter percentage so you validation set is larger and so more likely to contain an example of each stone tool. I do not know whether augmenting the images so cropping, rotating is valid for the tools but it would give you more data.

What an amazing use for Deep Learning. I assume you could have untrained people picking up stones in a site, have a app on their phone and classifying it as interest or rubble.

Regards Conwyn

2 Likes
4

Hi Ali and Conwyn,

Thankyou for the helpful suggestions, I’ll try my hand at implementing them. The limited samples mean I’ll probably try oversampling with some rotation or other augmentation.
And yes Colin that is one of the use cases I have in mind, or at least similar. A stone tool expert in your pocket could be very helpful for archaeological survey.