@rxwei found this swift forum mentioning trying to fix the problems listed in @jeremy’s blog post: https://forums.swift.org/t/generic-math-functions/21059.
Oh exciting! Thanks for the link
I have been pouring over swift for tensorflow Tensor, SharedArray, the proposal Vector2, Vector3, Vector4, numsw, Surge. There is really no consensus in the community for a solid multidimension array design for large scale numeric calculations with “transparent” support for multiple platforms CPU, CUDA, metal …
I’m really looking forward to help develop and validate a solid concept.
Has anybody here tried Metal computations, as one does with CUDA? I guess Apple uses it inside of CoreML.
What do you want to know about Metal compute?
Apple provides a bunch of building blocks in the MPS library (Metal Performance Shaders). This is similar to cuDNN and contains convolutions etc. This is also what Core ML uses to do its thing (although Core ML also has a CPU backend and another one for the Neural Engine).
If you write code with MPS or Core ML, often you’ll have to write custom Metal compute kernels by hand to fill in missing functionality that MPS doesn’t have. That’s more like CUDA, but not necessarily as flexible.
(What would be nice is something like Halide with a Metal backend so that it automatically compiles high-level descriptions of compute into optimized Metal kernels.)
Exactly. Which hopefully stuff like polyhedral compilation in MLIR can help move towards. And/or TVM etc…
Thank you for the response! As soon as I’ve never worked with it, I am glad to know at least something
I feel like taking into account S4TF it sounds like an opportunity to get into some “low-level” stuff. I was always interested in how things like PyTorch are implemented, like their basic blocks. But C/C++ CUDA/cuDNN is a bit too “low-level” for my skills.
Though I guess this thing you’ve mentioned, Halide, is something like high-level language to build shared, right? Independently from backend language.
Maybe I am wrong but numerical Swift seems to be a great opportunity for someone who doesn’t have a good understanding of C++ dive deeper into high-performance computations. Python is a cool thing and in many cases does it work pretty well but sometimes its limitations start to push me into a direction of compiled languages.
In theory yes, but I don’t think we’re there yet. I build ML into iOS apps so I write a lot of Swift but for any numerical computing routines I prefer to drop into C.
You need to use pointers for speed and the pointer API in Swift is horrendously complex and verbose. In addition, Swift’s memory management can really slow things down.
In some code I wrote recently in Swift, about 70% of the time was taken up by matrix multiplications (using BLAS) and 30% of the time by Swift’s memory management (ARC doing retains/releases etc). By switching to C the code became simpler, easier to read, and now 100% of the CPU time was available for the matmuls.
That said, proposed improvements to Swift’s memory model should go a long way to fixing this issue.
Yeah, I’ve meant – potentially
Up to the moment, it seems like there is no anything more ubiquitous and widely-adopted then C-based code and libraries. Would be great to bring a bit more modern language here.
If you look at how I did things in BaseMath and SwiftyMKL you can see how to avoid all that overhead - I got the same speed as C, and sometimes even faster. The code also ended up quite a bit cleaner than C.
Yeah, the post about BaseMath looks very promising! It is definitely worth to make a try and use it at least for some toy project maybe.