@jeremy In terms of architectures which can be implemented - My understanding is Keras functionality is a subset of Pytorch’s, although it might be simpler and fewer lines of code in Keras for some problems. If that is accurate, based on your experience, can you share some details about the problems where it would be best to use Pytorch instead of Keras?
I ask this to evaluate (in my case) how important it is to prioritize learning Pytorch over other things.
We should probably make this an ‘AMA’ in the next class!
Keras is, perhaps, best thought of as some pre-built modules that can be stuck together in convenient and flexible ways. Some things get a bit messy if they don’t easily fit, such as the GAN and neural style transfer stuff we did.
Pytorch tends to be less messy for that kind of custom stuff, and more importantly, far easier to build and debug, since you don’t have to first create a computation graph, then run it all at once - instead you get results at every step.
For doing custom RNNs, keras isn’t a good option at all, as we’ll see tomorrow, so you really want to use pytorch.
Overall, unless you’re fairly sure all your projects fit within keras’ purview, you’ll probably want to learn pytorch. You can learn it in a weekend or two pretty easily, I’d say.
Totally agree! Afte using Keras with TF for building more complex models like GAN etc, I can easily appreciate how flexible it is to use Torch for building dynamic computation graphs like LSTM with Attention.
So far my feeling is that for implementing recent papers, Torch is a better fit. Not only that it is more flexible and dynamic, but also it is easier to debug. For more matured and tested models that are meant for production, Keras with Tensorflow backend is a great choice due to its transparent handling of distributed GPU computing.
The Torch tutorial on Lesson 12 Wiki shared by @Matthew can be quickly gone through in half an hour or so, which proves @jeremy’s point that it is not that hard to learn.
Oh sweet!
If you folks started this initiative, I’d be happy to help and make a thread to provide the resources I have found, since I just started with pyTorch. (I personally didn’t see the need until now, though I have used Torch heavily before, aside from other frameworks)
