I’m looking to apply AdamW to training resnet18 on Imagenet from scratch. I’ve been looking around for some existing implementations, but so far I can’t find any dealing specifically with training imagenet using AdamW from scratch?
-The recent fast.ai “imagenet in 18 minutes” work used SGD. I imagine this was probably done for speed & simplicity
https://www.fast.ai/2018/08/10/fastai-diu-imagenet/
-The original AdamW paper looked at a downsampled version of imagenet.
-sgugger’s original work and the fast.ai dawnbench work only use AdamW for cifar or for fine-tuning an imagenet based model to the Stanford cars dataset
https://www.fast.ai/2018/07/02/adam-weight-decay/
Have there been any successful applications of AdamW to imagenet or am I barking up the wrong tree here?