Exporting a model for local inference mode

[muellerzr (Zachary Mueller)]

In terms of light weight take a look here:

Speeding Up fastai2 Inference - And A Few Things Learned

So today I went through and looked at trying to do inference efficiently. First thing you’ll notice is that there will be a decent gain in speed here, the main reason I believe is because fastai is attempting to do a lot on the back end and make other various functions/utilities easy to use and try to do it as efficiently as possible which may show the lack of speed. (also if I am wrong here Jeremy or Sylvain please don’t hesitate to call me out ) With that out of the way, let’s g…

On CPU single image inference I could bring it down to 732ms via the techniques mentioned.