I am trying to build a simple classifier that’s looking at ECG data and classifying them as normal vs. atrial fibrillation. Trying to implement this paper (https://www.ncbi.nlm.nih.gov/pubmed/29291535)
I first started by modifying the lesson 1, cat vs dog code to do the classification. However, I am having a really weird issue:
My accuracy is unstable and the validation loss jumps all over the place. The model is clearly overfitting. I have tried using weight decay (0.001,0.09) and higher drop out (0.6,0.8) to reduce this problem, but with no result. I think the fact that the validation loss comes down so low, is promising but how can I get it to generalize to the validation set?
I even used a simpler architecture, trying to replicate the one in the model and had horrible accuracy there. I have no idea what else to do, so any new ideas are appreciated.
I have 20323, labelled images in training set, 4088 for validation.
Unfortunately, that’s a confusion matrix from a previous attempt at the model where my validation test set wasn’t properly selected. My accuracy on the validation set in this model is less than 50%. Keep in mind since this is single label classification, that means it’s worse than random.
what if you try at first smaller learning rate and train let’s say 10 epochs?
line 78: lr = 1e-4 or 1e-3
line 79: learn.fit(lr,10)
if one epoch is not too time consuming you could try this line instead of line 79:
%time learn.fit(1.5, 1, wds=1e-4, cycle_len=15, use_clr_beta=(15,20,0.95,0.85))
I’m not sure what the images look like in ECG data, but i’m guessing that they don’t look like normal real world images. The cats and dogs example does transfer learning using the imagenet trained model, and none of the images look anything like the data that you’re now showing the model.
You can either try to find a model that’s trained on medical data as your starting point for transfering, or if you can’t find one try unfreezing the majority of layers in the network as the learned representations of the model aren’t likely to help.
