Adam optimizer more in nlp rather than in image classification?

Part 1 (2018)
1

Is Adam more used in nlp field?
and Adam has its recommended param values, so we don’t set lr when using Adam?

1 Like
2

Adam hasn’t been used much in NLP - I’ve seen papers that claim it doesn’t work well. However I’ve found it works quite nicely! And with @anandsaha’s new optimizer, it should be even better.

You should still use lr_finder to find the best LR.

1 Like