Create standalone binary for desktops, mobiles by Keras

Deep Learning
1

Do anyone tried to do that?Is it possible?I saw some posts say we can use TensorFlow to create standalone app for android, that is all I can find.

Ideal solution is convert the model into c or c++ api(these two languages can run on almost any os, hardware), in this way I can

1 : Call the api from Qt5, a library which could cross a lot of platforms(windows, mac, linux, android, ios)
2 : c, c++ can work with different languages, almost every high level language provide their solutions to communicate with c, c++ api.

ps : dlib is good enough for image classification, but it do not support functional api like keras do

1 Like
2

I forgot to mention, we can use dnn module of opencv3.3 to use the model trained by TensorFlow, caffe1 and PyTorch.

Problems are

1 : Keras do not in the list(hope it could call the model of keras build by TensorFlow backend).
2 : Not all of the layer are supported(unless I do not see Embedding)

This maybe the best way I could find right now, do you have another recommendation?

Edit : Not all of the models can be converted

3

I realize that this will be more specific than what you are asking but have you looked at Apple’s Core ML framework? (here is a worthwhile overview video)

The reason I like Apple’s solution is that Core ML allows you to easily integrate trained models into any macOS/iOS application. You would train your Keras model wherever you like (Ubuntu, etc.), save it and then import into your app. From there you can very easily use your model for prediction.

Again, this obviously isn’t as cross-platform as you seem to desire but it might be a good way to test our your model(s) on a mobile device very quickly. (plus I used to be an engineer at Apple so I am a little biased) :wink:

1 Like
4

Thanks, I will consider this solution if I only need to target mac and ios.

I tried to load keras model by opencv dnn module, it can load, but fail to do the forward call, I would switch to PyTorch in the future. If not the video of lesson9, I think I would prefer TensorFlow rather than PyTorch. I am very surprise a library with horrible api design can gain huge success in today, power of money + advertisement are awesome.

Horrible design of TensorFlow api tell me one thing, it is very hard to find someone good at complicated academia theory and software development even for a giant like Google.

1 Like
5

Now TensorFlow have experimental feature allow to compile your model: https://youtu.be/kAOanJczHA0

1 Like
6

I tried to compile tensorflow on my ubuntu several weeks ago, what I would like to say is, this is not a pleasant journey, absolutely far from good, over complicated

  1. Dependencies are horrible
  2. They rely on bazel, IMHO, this tool do not as mature as cmake, new versions may break backward compatibility and pop out lots of cryptic messages
  3. The c++ library of tensorflow have weird bugs, like can’t work well with opencv, need to use some work aournd in order to load the model successful etc
  4. Api, as always, they are over complicated, I have no idea why a company which gather the most brilliant engineers in the world fail to design elegant api, do google want to tell me those api are designed for super smart engineers like google?

These are the cons of tensorflow I meet, tensorflow have lots of pros out there, but it actually is not as good as other mentioned. The weirdest part is, seldom see anyone complain about the cons of tensorflow.

7

I understand your righteous anger. But I’m ordinary developer same as you, not a google engineer.
If you know good alternative for Tensorflow compiler please share it with community.

8

I am sorry if my words make you uncomfortable, I was not complaining to you but the library itself. There are different alternatives, but none of them can replace tensorflow completely.

  1. dlib, easy to compile, great performance, nice api and document, but lack of features, pre-trained models, do not support rnn.
  2. opencv dnn module, do not have decent gpu support(in fact, support of gpu is very poor in 3.3.1)
  3. yolo v2, if you need object detection + you only aim for windows and linux, this is a good solution
  4. Use pytorch to create you model, convert it to caffe model and load it by opencv dnn, as 2, this solution do not has solid gpu support

Unless my jobs must rely on tensorlfow, else I wouldn’t consider this library, their over complicated dependencies management is a pain to deal with, not to mentioned their c++ api are not yet mature, who knows what kind of bugs/break would introduced in new versions?