Adding new labels for classification

Let’s assume we have a classifier built on a conv network and a softmax layer which can classify for 10 labels of same type. For example, the labels are nations and the samples are the faces of people from different nations. Please also assume that the samples within a nation are very similar to each other.
what should be done if there is a new label ( new nation ) to be added to the softmax layer? I want to evolve my NN, in case of low probabilities at softmax layer with no outstanding winner among all outputs. In such a situation what could be done to evolve the NN to automatically identify a sample as a new label and increase the number of labels accordingly?

1 Like

I have an idea, but am not sure if it is a good one. I would like to discuss it with you.

Assuming we have trained the NN already for 10 labels and a sample doesn’t have a satisfying probability at softmax as it is lower than a predefined threshold. We can suggest that the sample belongs to a new label. In this case we pop the softmax layer and append to NN a softmax with 11 outputs and initialize the weights with the previous popped softmax’ weights. In addition to that we train the softmax layer with the new sample.

Is this a nice way of solving this problem?

That could work - although I suspect this is the best approach: https://papers.nips.cc/paper/5204-devise-a-deep-visual-semantic-embedding-model . We will learn about this in detail in lesson 10, once part 2 is out. :slight_smile:

2 Likes

@jeremy thank you very much for the suggested paper :slight_smile: It sounds part 2 will be even more exciting than part 1.

1 Like

Was about to ask for the same question. Thanks for both the question, the answer, and the excitement for part 2 of the MOOC. (maybe both wasn’t recommended)