Super interested in this - was playing around with the new PyTorch CPP features. If anyone’s interested, here’s how you can compile your model from lesson 1:
learn.load('stage-1') \\ or whatever your saved model is
example = torch.rand(2, 3, 224, 224) \\ dummy batch size, n_chanels, w, h
learn.model.training = False \\ disable dropout
learn.model = learn.model.cpu() \\ move to CPU
traced_script_module = torch.jit.trace(learn.model, example)
traced_script_module.save("model.pt")
This saves the model.pt file that you can use in C++ code to build a binary as shown in PyTorch C++ Documentation. You’ll get warnings due to dropout being in the model, but it should still work.
As a side note, the example worked for me out of the box without needing to use XCode or VSIde or some other C++ nightmare. Just remember to upgrade your cmake using
conda install -c anaconda cmake
I still have a ton of questions about this though:
- now you have your compiled model - what are best practices? Call your binary from whatever backend you’re using?
- data pre-processing: do it from within the C++ binary using PyTorch
dataor move it to your backend - pros and cons?