Starting next week on Friday, July 21st, the longest running, continuous fastai online study group will begin reading modern LLMs papers, starting with Language Models are Few-Shot Learners (the GPT-3 paper). We will then work our way through the seminal LLM papers to the present, covering one paper a week.
Our study group is coordinated through the fastai discord in the #cluster-of-stars text channel and meets weekly on Fridays at 2300 UTC (7pm Eastern) in #fastai-study-groups voice channel.
After reading the foundational papers, the plan is to work our way through all the modern LLM methods mentioned by Andrej Karpathy in his The State of GPT talk, along with any new developments since then.
The only prerequisite is that you are familiar with the basics of NLP, including Transformers, Tokenizers, and language model training or fine-tuning. See you in a week and a half!
Saturday, July 22nd, for those east of the prime meridian. ↩︎