I’m not sure that’s true. The PyTorch kernel seems much more cleanly designed, and as a result seems to generally stay a bit ahead in terms of capabilities vs TensorFlow. Whilst TF from the outside appears to have a bigger ecosystem, once you actually scratch the surface trying to use much of that ecosystem turns out to be impractical and/or clunky.
Perhaps if you’re inside Google things are different. But in the world outside Google everything from the poor installation experience, to the lack of true programmability of TPUs, to the weak design and implementation and ecosystem of XLA, to the frequent announcements followed by failed delivery of TF projects at the dev summits, and especially to the massive amount of code repetition and tech debt inside the TF code itself, it seems like TF comes with a lot of baggage.
I expect the Swift for TensorFlow team to succeed and create something better than either PyTorch or Python TF. But if they do, it will be despite TensorFlow, not because of it. E.g. MLIR, which @clattner links to above, will (hopefully) allow S4TF to largely extricate itself from the TF baggage and forge its own path.
OTOH, PyTorch has to deal with the baggage of Python, which was not design with today’s parallel processors or data science in mind, and as a result the PyTorch devs have to do heroic things to make stuff like JIT that works around some of these foundational problems. I think in the long term this won’t be sustainable.
So whilst I think S4TF can, over time, work around and/or bypass the TF baggage, I’m not convinced PyTorch can do the same with the Python baggage. For the next couple of years at least, I’d expect PyTorch to remain the first choice for data scientists that just want to get stuff done, but I don’t think that will remain the case in the long term.