Transfer Learning and Attention Network

Part 1 2022
1

Hi all,

Hope you all are doing well!

Please can you point me to the best resources for introductory learning to:

  • Transfer learning; and

  • Attention network

I know some of the above were covered in this course, including, for example, Transformers, but am looking for introductory resources to learn the above for a beginner.

Thanks in advance. Much appreciated.

Kind Regards,
Zakia

3 Likes
2

Hi Zakia,
I think the course covers pretty much all you need to know about Transfer Learning especially the version of the course before this year (2022) one where Jeremy went over the article describing transfer Learning applied for the first time on ImageNet. Also, Jeremy’s work on UMLFit which exploits the transfer Learning concept, has inspired latest major breakthrough in NLP and the underlying principle of this work is explained in the course.

For transformers, I have not yet watched this year course about it but this resource can be interesting: https://fullstackdeeplearning.com/spring2021/.
In the Week 4 section, you’ll see a dedicated lesson on transfer learning and Transformers (even coding it from scratch to understand the nitty gritty details).
Best of luck in your endeavor.

1 Like