Maybe, I’m asking sth which has been questioned similarly many times, but I can’t find an exact answer to my question, so I’m asking one more time here.
Throughout this course, fastai is used instead of Keras+TensorFlow. At the FAQ of this forum, it also mentions why it is in this way, say the limitation of TF + Keras (they are much slower, result in less accurate models, and take more code)
My question is, is there limitation for fastai ( <1> on its own <2> in comparison to TF + Keras) ? If fastai is better than TF + Keras in all areas, can we forget TF + Keras at all ?
I hope that this question, and the coming discussion can let us have better understanding on both fastai and TF + Keras. Thanks !
I think the biggest issue with fastai, or even standalone pytorch, would be production deployability.
Another (implicitly mentioned in your list), is that skills in Keras/TF could be much more easily marketable. And that’s very important for those of us that would be really happy to switch to a DL job.
My solution (not particularly original) is to follow along with you and at the same time keep studying Keras/TF. For example, in my opinion Chollet’s book is an ideal complement of fastai.
Could you elaborate on the production deployability aspect?
From what I have learnt from folks such as James Bradbury (Salesforce) and other engineers at Tensorflow, they ack that while pytorch code deployment is non-trivial, it is not entirely impossible. There are three options for us : one, convert Pytorch code to Caffe (most preferred technique at Facebook), two, use ONNX and three, once your prototypical phase in Pytorch is done – convert the code to tf.eager. I got the idea that the differences between the tf.eager and Pytorch are slimming down since either code might look quite similar at kernel level.
Not sure if I am missing anything here.
Most people just stick pytorch straight behind a flask end-point or similar with CPU inference. It’s only likely to be an issue if you need to scale really big (in which case you can afford to hire specialized engineers!) or you need to deploy to mobile.
I don’t think the marketability issue is a big one, myself, since switching languages/libraries is pretty straightforward (a couple of days of study) once you understand the concepts. But I think trying to learn DL in a library that doesn’t show best practices, and hides concepts behind layers of complexity, is a big problem!
Having said that, Chollet’s book is excellent and reading it after or at the same time as doing the fast.ai courses is a fine idea, as long as you don’t find it distracting from the concepts we’re teaching.
Core limitations aside and the many layers of unnecessary complexity (which I think “eager” now resolves for TensorFlow) what best practices is the TensorFlow library lacking in?
Learning rate finder, discriminative learning rates, batchnorm freezing, NLP classification on large documents, use of continuous and categorical columns in tabular data, access to CSV files, etc etc etc…
Regarding deployment, I was working on ONNX export for fastai, and got it working pretty well for the standard image models, but it is hacky.
I would like to start a community effort to add a onnx export functionality to fastai if people are interested and no other projects like this are currently running.
I will write a message tomorrow going into much more depth about what I found.
DL framework (either fastai or tf) is still under rigorous development, either one is near to be very mature. TF version 1.0 was released in 2017, not a very long history. There is a long way (or war) to go. Can we say in this way ?
I’m new to deep learning and was doing this course. Something i believe you said in the course was that using another library such as tensorflow would be much more easier to screw up. From that perspective, wouldnt u also say that knowing how things operate under the hood is important in building a serious model?
fastai might be easier to use, but as a beginner who wants to build their own predictive model, if i require my model to be accurate im going to need to know how every single aspect works under the hood and how parameters must be tuned-- maybe im wrong but to get down to a more granular level isn’t the heavy lifting that other libraries require something that shouldnt be ignored?
We (Prolego) switched to pytorch/fastai for our consulting engagements for many of the reasons @jeremy lists. Additionally we find that enterprise clients are (understandably) wary about taking over whatever we create for them.
When we start the conversation with “Pytorch is a python class …” the leadership gets much more comfortable.
Most Fortune 500 companies want to compete on their data, business model, expertise - and not AI innovation. So we help them apply AI to what they already know how to do.