I have had this problem for a while, and never thought about asking here… why not?
I need to use some models (Convnet VGG-ResNet style)I trained in Python (either with Keras, fastai, tensorflow, cntk) for inference in a C++ production environment. The requirements are basically:
- Speed (inference in milliseconds)
- Small size of the library and its dependencies.
The approaches that I found are two, mainly, one the opposite of the other:
- Use the original APIs in C++ of the respective model (pro: speed, con: huge dependencies)
- Rewrite from scratch Conv and Dense layers (not so speedy…)
Had any of you somehow magically solved these issues?
Hoping for a feedback, thanks!