Best model to use for age detection?

I’m trying to create a model for age detection with imdb cropped and wiki datasets. I noticed that my train loss is not reducing at all if I use pre trained resnet50 with imagenet_stats. I tried running in without pretrained, using learn = create_cnn(data, arch, metrics=error_rate, pretrained=True).to_fp16()
I have total 107 classes (age 1-107)
Nothing seems to work. Any suggestions??

A regressive classifier.