Transfer learning for Recurrent models, not just CNN models?

The first lesson using transfer learning to finetune VGG16 (previously trained 1000 objects of imagenets) to predict cats and dogs, is amazing.

I can imagine the implication that the common people can get a lot of done by just focusing their limited resources on studying how to finetune models like VGG16 which are expensive results produced by large resource-rich research institutions.

fast.ai course has demonstrated brilliantly transfer learning can be applied to CNN models. The immediate question come to mind is, can transfer learning be applied to all fantastic models, for example RNN models as next step?

Thanks
@rachel @jeremy