I am using Part 2, Lesson 10 ULMFit concept to classify text. However, unlike IMDB data, I have 4 classes. I have an evaluation set for which I have no label. My end goal is to run the trained classifier on this evaluation set. My question is
after the learner.fit() step how to go for prediction? I mean will learner.predict() work? What’s the syntax?
How to feed the evaluation data into the learner.predict() and get the corresponding label?
In the lesson, the learner.predict() is not used at the end. So I am looking for the correct syntax for predicting the text label with the trained learner.
It will be really helpful for me if someone can point me to the right resource.
I am trying something similar. I have a dataset with 131 labels ranging between 0 and 472. I have trained the classifier and accuracy as returned by learn.fit() after 14 epochs is 0.78. To cross check this accuracy, I used learn.load() to load this model and then used learn.predict() which returns a 2D array of numbers (one for every label for every example). Using ‘np.argmax’ I get the predictions. But when I use the sklearn ‘accuracy_score’ to measure the accuracy I get 0.02. Can someone please explain to me what I have done wrong? And also, how exactly is the accuracy calculated? I have looked at the metrics.py code but do not completely understand it.