I think the course covers pretty much all you need to know about Transfer Learning especially the version of the course before this year (2022) one where Jeremy went over the article describing transfer Learning applied for the first time on ImageNet. Also, Jeremy’s work on UMLFit which exploits the transfer Learning concept, has inspired latest major breakthrough in NLP and the underlying principle of this work is explained in the course.
For transformers, I have not yet watched this year course about it but this resource can be interesting: https://fullstackdeeplearning.com/spring2021/.
In the Week 4 section, you’ll see a dedicated lesson on transfer learning and Transformers (even coding it from scratch to understand the nitty gritty details).
Best of luck in your endeavor.