I picked a simple face classification as a starter project since i figured that would help me learn fastai principles faster than trying to deal with a too difficult a classification problem. I have FaceA and B sets each with 700 images each.
The trouble is with just 4 cycles it has an error rate of zero between face A and B and the learning rate plotters don’t plot anything. It was disappointing because now there’s not much for me to play with in the notebook lesson which is how i learn.
On the other hand it seemed like a nice problem to have but then I tried to test it in prod mode with other faces and found it incorrectly classified all of them into A or B. I then thought maybe it needs a nonce class to act as a dumping ground for any non As | B’s so i created one with random images in it. It does better this time but still misclassifies a lot of faces into A or B instead of nonce.
So what to do here? Are these outcomes symptomatic of some known cause.
- Do i need more variety in the training set?
- Is there a probability/certainty value i can refer to / tweak somewhere?
- Do we actually need a nonce class for anything not A or B or is there some other built in way of rejecting these - since a nonce class could effectively be infinite in size.
- What do we do when training has zero error but production has lots of bad predictions? Do i just keep on training? Seems wasteful.
Thanks