I have been trying to follow along with the suggested reading materials with the lessons like this one on Exploring the limits of language modeling. But it happens quite often that I come across a lot of things that I have no idea what it is. Quoting from the mentioned paper
Count-based approaches (based on statistics of N-grams)
typically add smoothing which account for unseen (yet possible)
sequences, and have been quite successful.
I have no clue what “count-based approaches”, “statistics of N-grams”, “add smoothing” are. I understand that the simplest thing to do is to google the terms. But while reading the papers there are references to many other papers and many such terms. And if I search for them it could become recursive quickly. Seems that would be a bad way to proceed to fill the gaps in my knowledge.
Is it just me or do other people face this issue too? What do other people do when they are faced with this issue? Any advice on how I could proceed? I understand that fast ai is a top down approach and we shouldn’t be bogged down by details. But at some point you have to do that, right? Else my knowledge will be very superficial. No?