Transformers architecture

Hello everyone!
First of all thanks to the FastAI team and the community for the fantastic work, I have learned a lot thanks to your efforts.

I was wondering if in the future the NLP part of the Course will also be extended to the concepts of Attention mechanism and Transformers architecture.
Thank you for any input you’ll give me!

Hi @janetbran, if you mean if I got a response, no :slight_smile:
Actually there was no problem to solve at all, just my curiosity…

Based on what I have heard from Jeremy, the next version of the deep learning course will most likely cover Transformers.

2 Likes

Thanks for the feedback @ilovescience!