My accuracy is unstable and the validation loss jumps all over the place. The model is clearly overfitting. I have tried using weight decay (0.001,0.09) and higher drop out (0.6,0.8) to reduce this problem, but with no result. I think the fact that the validation loss comes down so low, is promising but how can I get it to generalize to the validation set?
I even used a simpler architecture, trying to replicate the one in the model and had horrible accuracy there. I have no idea what else to do, so any new ideas are appreciated.
I have 20323, labelled images in training set, 4088 for validation.
Unfortunately, that’s a confusion matrix from a previous attempt at the model where my validation test set wasn’t properly selected. My accuracy on the validation set in this model is less than 50%. Keep in mind since this is single label classification, that means it’s worse than random.
I’m not sure what the images look like in ECG data, but i’m guessing that they don’t look like normal real world images. The cats and dogs example does transfer learning using the imagenet trained model, and none of the images look anything like the data that you’re now showing the model.
You can either try to find a model that’s trained on medical data as your starting point for transfering, or if you can’t find one try unfreezing the majority of layers in the network as the learned representations of the model aren’t likely to help.