I trained a classifier model with the NLP tutorial for chapter 10.
Now I want to predict and save the label for over 1.000.000 texts.Right now I am doing this for every single text with the load.predict(inputstring) function. It takes to much time.
Can I optimize the performance?
You could use batch prediction like this:
test_dl = learn.dls.test_dl(test_df) preds = learn.get_preds(dl=test_dl)
Note that this rearranges the order of the input texts behind the scenes, so you’ll need to reorder the predictions using
.get_idxs() on the test dataloader.
Or you can have a look at fastinference which sould speed up
learn.predict() quite a bit.