Thx a lot of sharing this - this is super valuable information!
For running inference using Neural Engine, what do you think is an okay number of parameters for the model to have? I think some of the smallest models out there for vision have ~5mln params (nasnet mobile) but their arch can be more tricky to implement (and inference runs quite slow on CPU from my experience) and resnet-34 has over 20 million parameters - not sure if that wouldnât be prohibitive for mobile hardware at this point in time? There is also the question of weights - not sure what the saved weights sizes are for resnet-34, guessing maybe around 150mb.
I am aware there exist methods out there to reduce the used parameter count but all of them increase the amount of work necessary.
But you need to implement all the gates aahhhh. Again, can be done (I saw fastai implementation of LSTM cell floating around somewhere IIRC - can be quite helpful I would believe) but would require more work.
I wonder about shipping text / images to an API endpoint for inference. Is this a pattern that is used quite often? (I understand it comes with its own headaches, scaling issues and costs but I wonder what would be the approach for rapid prototyping?)
There used to be this quite famous style transfer app from some time ago⌠donât remember the name, but from what I remember reading back then was that they were shipping off images to run the models on their servers? I wonder if this is the general approach taken or if people use the processing power available on the phone.
My rule-of-thumb is that 20M parameters is roughly the upper limit. You can run larger models too, just not in real-time. But it also depends on the model architecture. Some operations are faster than others.
The biggest factor, however, is the size of your inputs. A model that works on 256x256 images will perform a lot faster than on 512x512 images, even though the number of parameters stays the same.
BTW, implementing the logic for LSTM isnât that hard. You can combine the weights for the different gates into two different matrices, so it only ends up being 2 matrix multiplies, a tanh, and a sigmoid. I also wrote a blog post about that. https://machinethink.net/blog/recurrent-neural-networks-with-swift/
For doing style transfer, if youâre using the type where you have a network that is trained on a specific style (say, van Gogh paintings) then you can do this in real-time on the device. No need to send images to some server.
If youâre using something like Googleâs ML Kit and you want to use the more advanced features, it will send the images to their own servers and then return the result to your app, but this is all handled behind the scenes.
Thanks for the many useful resources in this thread. I just wrote up my experience setting up S4TF on my Ubuntu machine and posted it on Medium. Thought Iâd share it here in case anyone wants to follow suit. I address a few potential points for troubleshooting, so I hope itâll be helpful to some. FYI: this is the âfriendsâ link, so you donât have to be a paid Medium subscriber to read it.
Thanks for the suggestion! Simpler is better⌠Seems I was robotically following the Swift-Jupyter README last night and solving problems that didnât need solving if you could avoid creating them in the first place.
Itâs actually simpler still to create your conda env with python==3.6 specified and go from there. Edited my Medium post to reflect the new approach.
FYI prompt-toolkit > 1.0.15 has incompatible issues with ipython 5.5.0 and prompt-toolkit < 2.0.1
has incompatible issues with jupiter console 6.0.0
Found an issue this time with the requirements.txt file. The entry for prompt-toolkit==1.0.15
raises an issue on install jupiter-console 6.0.0 has requirement prompt-toolkit<2.1.0,>=2.0.0, but you'll have prompt-toolkit 1.0.15 which is incompatible
To this vague message I tried to install prompt-toolkit=2.0 this gave error that this version does not exist and gave all the existing version.
Version from 2.0.1 ⌠2.0.9 exists but 2.0.9 is incompatible with ipython 5.5.0 so is version 2.0.1.
Guess thatâs the flux that swift for tensor flow is in
This is an issue with Swift-Jupyter, but it doesnât seem to be a breaking incompatibilityâŚunless it relates to the issue Iâve been having with Python interoperabilityâŚ
Iâve addressed this in the latest update to my Medium post, which contains other updates, too. Guess I shouldâve waited to publish until after I had tested out more than the first *.ipynbâŚ
The conda instruction you reference above was a copy-paste error. But it turns out that adding those libraries doesnât help you. Iâve added another step to the setup in my post that addresses the issue of getting GPU functionality.
Finally - back to the Python interoperability issue - Iâve been unsuccessful in getting Python imports to work. I think itâs somehow stemming from the following: in python_interoperability.ipynb, the call import Python imports Python2.7 instead of Python3.6.
I didnât use the requirements.txt - I donât like that it pins all those specific versions. I donât think itâs a good approach. IIRC I already had all the prereqs in a standard fastai conda environment.
@saeta Perhaps you could update the install docs to remove these issues:
PYTHON_LIBRARY env var
conda env approach
Remove pinned versions from requirements.txt
Whilst pinned requirements are great for production situations, they can be troublesome for folks looking to use current versions of stuff and to use their existing environments where possible. In this case, my existing env worked fine without any of the pinned versions, which is handy since I didnât have to set up my notebook configuration again, and I can easily switch between kernels.
This got me on the right track. I now have Swift-Jupyter working via a streamlined approach using conda, as described in my reply to the other thread: How to Set Up Jupyter Notebooks for S4TF on Docker
I can remove all the pinned versions from requirements.txt, and add a section about how to install using conda!
Not sure what you mean by the âPYTHON_LIBARY env varâ issue. Do you mean that we should suggest adding ââswift-python-library â to the âregister.pyâ command in the install instructions, like neuradai does in his medium post? Thatâs the only PYTHON_LIBRARY env var related thing I see in this thread.
A few weeks ago I had to set the PYTHON_LIBRARY env var to get py3.7 working. But I suspect --swift-python-library didnât exist at that point so perhaps my info is out of date.
I am a little confused I have swift notebooks set up except that I have an issue.
To try out my installation s4tf I have cloned from git hub the tutorial directory in which there are several notebooks I have opened two of these
A Swift Tour
and
custom_differentiation
In the first notebook the first code cell works but the second and third fail with a syntax error
see these images
First cell
print(âHello world!â)
second cell
var myVariable = 42
myVariable = 50
let myConstant = 42
File ââ, line 1
var myVariable = 42
Syntax error: invalid syntax
