Language modeling question. Lesson 4

Few question on language modeling from lesson 4.

1.While testing a trained language model, why do we need to reset the hidden states? What exactly the reseting is doing here?

Reset hidden state


  1. What exactly is the prediction here

Get predictions from model

res,*_ = m(t)
When i look at the shape of res it says
torch.Size([21, 37392])
What is 21 here?

Thanks for your help

1 Like

I also have trouble finding what reset does on the forum. Hope someone can point us to some references.