[New PR]: Add ReduceLROnPlateauCallback

As per the new PR template, I wanted to describe the new functionality I pushed to the repo.

This adds a ReduceLROnPlateauCallback. It inherits from TrackerCallback and uses similar naming.

This callback monitors a metric / loss of your choice. If it does not see an improvement on the metric for patience amount of epochs, it reduces the lr by specified factor.

This would not be useful for training with the one cycle policy, but I think it is still a useful callback to have. In particular, when working on a new problem or trying a new architecture, using this callback combined with the early stopping one, we have something that we can let loose and it will produce good results and give us a bit more insight into the training regime that might be appropriate. Once we have this baseline result, we can move onto one cycle and or cyclical annealing. Don’t think this is applicable across the board but when looking to train the model as best as we can (without regard for how long the training might take) I think this is a very useful callback to have and fits really well with my workflow.

zombie voice Teeeest, we neeeeed aaaaaaaaa teeeeest.
Thanks for the work otherwise, just merged it in master.

1 Like

Thanks for the PR! I guess the template wasn’t clear - what we actually want it to first discuss on forum, then create PR after agreeing about the approach. :slight_smile:

For future reference - we should ask for the test in the PR, and only merge once we have it.

@radek, thanks for this callback. I would only suggest two things to clarify the behavior:

  1. The LR will be reduced only while current LR is greater that the min_lr .

  2. Default value for min_lr = 0.001 appears too high, or I didn’t get your drift. :slight_smile: