Going through: https://github.com/fastai/fastai/blob/master/courses/dl1/lang_model-arxiv.ipynb
Looking at the sample model method:
def sample_model(m, s, l=50):
t = num_str(s)
m[0].bs=1
m.eval()
m.reset()
res,*_ = m(t)
print('...', end='')
for i in range(l):
n=res[-1].topk(2)[1]
n = n[1] if n.data[0]==0 else n[0]
word = TEXT.vocab.itos[n.data[0]]
print(word, end=' ')
if word=='<eos>': break
res,*_ = m(n[0].unsqueeze(0))
m[0].bs=bs
Does the model remember previous words when going through sample model loop as it is generating the words? Does it some how remember the previous state until reset is called? If so where is that state being remembered in?