Does eval mode of a nlp model remember previous state?

Going through: https://github.com/fastai/fastai/blob/master/courses/dl1/lang_model-arxiv.ipynb

Looking at the sample model method:

def sample_model(m, s, l=50):
    t = num_str(s)
    m[0].bs=1
    m.eval()
    m.reset()
    res,*_ = m(t)
    print('...', end='')

    for i in range(l):
        n=res[-1].topk(2)[1]
        n = n[1] if n.data[0]==0 else n[0]
        word = TEXT.vocab.itos[n.data[0]]
        print(word, end=' ')
        if word=='<eos>': break
        res,*_ = m(n[0].unsqueeze(0))

    m[0].bs=bs

Does the model remember previous words when going through sample model loop as it is generating the words? Does it some how remember the previous state until reset is called? If so where is that state being remembered in?

Yes, the state is stored in the hidden attribute in the model. Take a look at the stateful RNN we built in class to see how that works.