Classify multiple texts

Hello,
I trained a classifier model with the NLP tutorial for chapter 10.
Now I want to predict and save the label for over 1.000.000 texts.Right now I am doing this for every single text with the load.predict(inputstring) function. It takes to much time.
Can I optimize the performance?

Hi Dominik,

You could use batch prediction like this:

test_dl = learn.dls.test_dl(test_df)
preds = learn.get_preds(dl=test_dl)

Note that this rearranges the order of the input texts behind the scenes, so you’ll need to reorder the predictions using .get_idxs() on the test dataloader.

Or you can have a look at fastinference which sould speed up learn.predict() quite a bit.