Here is a recent competition on kaggle as part of CVPR to identify 200 different food dishes. https://www.kaggle.com/c/ifood2018/
Basically the fastai multiclassification approach will work just fine, given enough images, enough for me to get 2nd place.
I did read one paper that approached the problem with a ‘wide slice’ resnet, since images of plates of food often have a context based on top to bottom.
To get to calories, you can build a library against those food dishes - pad Thai, chocolate cake, pizza, perhaps prompting the user for portion information.
It wouldn’t be too difficult to build such a model, just a few days for that comp. as always, collecting the annotated data is the grind and the value.