Very Large Number of Class Labels

I’m working on an image recognition project that will eventually need to be able to distinguish between 2000-5000 different classes.

I’m hoping to use a straight forward transfer learning approach, similar to those taught on the fastai course. However, I’m worried that I might run into problems due to the large amount of class labels.

Does anyone have any advice about training CNNs with large amounts of class labels? Am I naive to think I can do this using transfer learning with say a pre-trained resnet or inception model?

Any insights into this would be greatly appreciated.