Modern LLM Paper Reading Group: Fridays at 2300 UTC (7pm Eastern)

Deep Learning
1

Starting next week on Friday, July 21st[1], the longest running, continuous fastai online study group will begin reading modern LLMs papers, starting with Language Models are Few-Shot Learners (the GPT-3 paper). We will then work our way through the seminal LLM papers to the present, covering one paper a week.

Our study group is coordinated through the fastai discord in the #cluster-of-stars text channel and meets weekly on Fridays at 2300 UTC (7pm Eastern) in #fastai-study-groups voice channel.

After reading the foundational papers, the plan is to work our way through all the modern LLM methods mentioned by Andrej Karpathy in his The State of GPT talk, along with any new developments since then.

The only prerequisite is that you are familiar with the basics of NLP, including Transformers, Tokenizers, and language model training or fine-tuning. See you in a week and a half!

  1. Saturday, July 22nd, for those east of the prime meridian. ↩︎

8 Likes
2

Do you plan to record these?

1 Like
3

please record these sessions, it’ll be 4.30 am in India.

4

Unfortunately, most (if not all) of the sessions will not be recorded. The study group is pretty informal, and we’ve found that not recording helps with participation, willingness to ask questions, etc.

1 Like
5

If you are interested but couldn’t make it, this Friday, July 28th, we will be reading Finetuned Language Models Are Zero-Shot Learners.

Future weekly updates will be on the fastai discord: