I recently embarked on a journey to learn Tensorflow. Here is what I learned along the way
The things that are great about Tensorflow
- There is a lot of research level code written in TF by some really smart people. Reading the code you learn a lot about building DNNs all the way to the lowest of details. You also learn how people tend to think about structuring the code and learn about the abstractions that permeate the deep learning world. For instance, my guess is that the Stepper we know from fastai might have its ancestry going all the way back to the idea of global step in TF. The issue though is that the code you will find might be doing very interesting things, but is often tightly coupled together and it’s hard / impossible to reuse it. There is often also little in terms of documentation - you need to navigate through code to figure out what is going on.
- There are many gems that the engineering teams at google produce. Case in point is the new Dataset API (a very nice way of reading data sequentially of the HDD and still being able to shuffle it, etc, with a very nice functional interface). Problem is, the documentation seems like it was written by the product team looking after this particular piece of code. The most likely scenario a practitioner is to encounter is reading off files from disk. But nowhere in the docs (maybe I didn’t look right) could I find how to actually feed the Dataset API anything but data from a CSV! Than the code you need to find in the Tensorflow repo to understand what is going on is not very easy to parse. The Dataset API is very nice and polished but writing the TFRecords that are used as inputs has a lot of low level plumbing to it that might not be easily accessible. This could however be fixed relatively easily by more examples catering to practitioners vs what seems right now like documentation geared towards people coming across ML for the first time in their lives or Data Engineering pros spinning up tens of instances in a heart beat and streaming the data to them with relative ease. Having said that, if you find an abstraction or a piece of code that addresses the problem you are facing, you are likely to be in a very good spot.
- There are a lot of good things happening in the Tensorflow repository, such as the object detection API and the newly introduced estimators. They are good in the sense that if you need a job done, and if you manage to figure out how to use those specific solutions (what code to write, how to transform your data, etc) you can achieve great results not knowing anything about DL. If you are building a product, say a mobile app, maybe there is room for such off the shelf solutions such as the one that the object detection API provides.
In summary, I had a lot of fun writing whatever little Tensorflow code I wrote and there are many things that are great about the ecosystem. The outreach that google is doing via Kaggle competitions also seems to be working very well. If I were working with Tensorflow on a daily basis as part of an engineering team, I could both enjoy it and learn a lot through this experience. I feel I already learned a lot through whatever little exposure I had to the ecosystem.
Those were good times but I am going back to PyTorch. How you say things is secondary to what you say, and there is no other resource I am aware of that comes close to the materials shared by fastai.
In the webdev world there is this big debate between using Java or Ruby. You can do the same thing in either but the question is not about capability. DHH, the creator of Rails, has a philosophy that he wanted to create a framework that should he lose everything he has, a single person with a clue (like himself) could leverage the framework to bootstrap something amazing. And that is the sort of feel that PyTorch has to it, which is nice.
Anyhow, don’t want to go into a discussion on what is better but maybe some of the information above can be useful to someone. I learned quite a bit from looking at Tensorflow and will most likely use Tensorflow down the road, but it might not be for training custom models but rather using some of the prebuilt solutions or reading Tensorflow code to learn about some novel method through code.
As a side note, I don’t think anyone should look to Tensorflow for learning Deep Learning or Machine Learning. You need to have some fundamentals of both and be quite okay with code to find your bearings and understand what is going on. Maybe the materials such as udacity courses / coursera courses make this experience survivable, but I have a hard time imagining how they do it. (Haven’t tried so maybe I should not be speaking!).