I’m training an old model with a new dataset. I’ve tried a few learning rates and done around 10 or so cycles of training, each of 10-15 epochs. However, after the initial drop in the first cycle, my validation loss and error rate have stayed steady, no matter what I try in terms of learning rates, moms or wd etc.
Anyone know why? My training loss is still falling… does that just mean I need to train it more? I’ve probably trained (collectively) for 5 hours now.
This is the last run: