Hi,
As I study Ch. 10, I notice that the dependent variable is the independent variable offset by one.
i.e.
if x = ‘a b c’, then y = ‘b c d’, i.e. offset by one. I was curious to know why. What would be the issue with just having the next word as the dependent variable i.e.
x = ‘a b c’ , y = ‘d’
Hi Rajiv
Can you guess what word comes …
That is basic a language model to predict the next word give the previous one(s). Before the 1960’s childrens books for reading contained phrases which occured frequently in daily speech. Then teaching of phonics became popular and the old ways were scorned. Then the neuroscientists discovered are brains are actually design to predict the next word given the {you guess}.
Hence language models are simply predict the next word.
To quote Chomsky - Colorless green ideas sleep furiously. A sentence but not in the real world.
Regards Conwyn