I’ve been tweaking my learning rates after a few epochs using
model.optimizer.lr = new_lr and
model.optimizer.lr.set_value(new_lr). If I use the former my results after a few more epochs line up pretty well with Jeremy’s. If I use the
.set_value method, the “proper” way according to this github thread, I get different, much worse results for the same number of epochs.
Another method I haven’t tried is to use the Keras backend
k to update the learning rate:
Any idea what’s going on behind the scenes that would make these methods behave differently?