Language model accuracy

(Vignesh K) #1

So, I started watching the lectures and tried out the Language model notebooks and have a few questions. While fine tuning the language model on IMDB dataset. What does accruacy mean in the output? Is it the currently training LM model accuracy on val set ? How do I interpret the accuracy ? And also where can I find the accuracy of the original Wikitext103 model ?

1 Like

#2

Hi vignesh1905,

I am very interested to experiment with this model,however, I have modified the code to point to the course data set and receive this error:

Since it seems you have been able to run the code I hope you would be able to advise how I might resolve this error.

Thank you

alvu

0 Likes

(Vignesh K) #3

@alvu I think your path seems to be wrong. Did you check the path from where you are running your notebook from. If not try giving the full path.

0 Likes

(Serge Mankovski) #4

I am interested in figuring out meaning of accuracy in context of Language Model. This question went unanswered 9 months ago. May be someone can explain meaning of accuracy metric in context of Fast.ai Language Model learner now?

1 Like

(Serge Mankovski) #5

I think I’ve got it now. Language model is a classification model. Accuracy of prediction is how often on overage model predicts the right word from a given vocab. Hence accuracy metric in this case is equal to probability of predicting correct next word from a given vocab in a number of Bernoulli trials equal to the number or words of the corpus.

0 Likes