Language_model_learner returns different predictions everytime time

(Pavel) #1

Everytime I launch learn.predict("…") function it gets different prediction. Is it right? I thought that in preiction mode there is no any dropouts, so the result should be determinated…


(JamesT) #2

We don’t take an argmax in the predict function, we sample from the probability distribution we get when we run a softmax on the logits.

1 Like

(Pavel) #3

And what the reason for this? Could you give any sources where it’s already explained?


(JamesT) #4

In your Jupyter notebook, run


If you read the source code that they show, my previous statement will be verified.
As for why it is done, well there are a few ways of generating text. Taking an argmax at every time step is a greedy search, what learn.predict does is described to some extent starting from minute 15 in the following video.