I have to all this for loading the learner for prediction in the flask application because of load_learner() from .pkl file is not working. Please provide an elegant way of doing this for a single image. After getting the prediction, I am not able to convert it to numpy or OpenCV image format. And If I am trying to save the image, it is saving it a blank image. Please help asap!!
My issue is that I never came to data science (and from there here) via a programming background. I do know a lot of data science programming but zero webstack or cloud. All the deployment guides seem to assume you ‘know’ webstack and cloud and work with it regularly. I don’t mind at all learning these things but I don’t want to learn them in their entirety - just “webstack/cloud for deep learning.” Tools to ‘go faster’ will help me nothing because as I learned when setting up Google cloud there are many things that always ‘go wrong’ or have changed etc and these roadblocks will be worse with an efficiency tool. Rather, I need to know the high level hoops I have to jump through and a list of the things I need to understand or try to jump through them. Thanks.
After scouring the web and forums for two days trying to figure out how to do this I have now discovered fastai have guides to put models into production, which I never noticed before. Doh!
Thanks a lot for your reply. I completely understand and share your perspective, as I also a maths background but almost 0 experience with web/cloud dev.
The Fastai tutorials you’ve just found are incredibly helpful for putting your first model in production, so hopefully it may help you solve some of your problems. But in my experience, the process to deploy is still tedious (especially if you want to use AWS lambda or Azure Functions), and many more advanced stuff is not covered by the tutorials (for example, deploying a web API, deploying on mobile, etc…).
I created a complete example of deployment fastai model in c++ application. There are some parts of the process in the forum. But I think it would be useful to have all of it in one place.
I’m attempting to follow the aws_lambda guide. https://course.fast.ai/deployment_aws_lambda.html Upon initial training, I have 3 models and data totaling about 800 megs compressed, and 1GB uncompressed.
One of the steps is to use Torchscript to simplify/streamline the model. The example they give is with a CNN. I’m attempting to deploy a variant of the IMDB ULMFiT implementation that they worked with in lessons 3 and 4. Clearly, the deployment guide is set to trace out a CNN.
I’m new to fast.ai, and discovered that torchscript existed yesterday when I started trying to follow the lambda guide described above. I’m still trying to get a sense of torchscript… the “gentle” guides are still over my head at this point. I’m honestly surprised that torchscript didn’t make the cut to be included in the 12 lessons of the fast.ai course. That feels like a bit of an oversight.
Can somebody point me in a good direction to begin understanding how to modify the CNN script for an RNN? A code sample would be amazing, but failing that, documentation/training would be great.
Ran into this recently and found a solution from this blog post:
You’ll need to specify fastprogress as a hiddenimport in your spec file, specify the location of your hook files, and then create a new hook file with the following content:
from PyInstaller.utils.hooks import copy_metadata datas = copy_metadata('fastprogress')
Be sure to name it hook-fastprogress.py
For more information check out PyInstaller’s documentation regarding hooks and hidden imports.
In lecture 2, Jeremy said to try to build a web app “This week”, and said that there are lots of tutorials out there. From the two days worth of tutorials I’ve watched so far, I have gathered that in order to build a web-app, you need baseline understanding in the following areas:
A framework like Flask or Django
JavaScript
CSS
HTML
Template Engines
Maybe SQL too…
That’s not happening “this week”, at least for me. I feel like I am missing something. Can someone point the dumb new person to the right place?
The gap is not so big but you can use some accelerators which are compatible with Openvino (Intel GPU, Neural Compute Stick), and normally, it is cheaper than NVIDIA GPU
I’ve been trying to deploy a fastai model using lambda with no success.
As I was not able to use a public PyTorch layer, I’m building my own.
Applying all the recommendations, I ended up with dependencies totaling 437Mb. The compressed layer has 130Mb.
When trying to run the lambda function, I get the error ‘No space left on device’.
Yes I see, the problem is the pytorch dependency, which is too big. It’s why I used the separate public layer for pytorch.
This post by @matt.mcclean explains how the pytorch layer was made, it might help:
AWS Lambda has a limit of 250 MB for the deployment package size including lamba layers. PyTorch plus its dependencies is more than this so we need to implement a trick to get around this limit. We will create a zipfile called .requirements.zip with all the PyTorch and associated packages. We will then add this zipfile to the Lambda Layer zipfile along with a python script called unzip_requirements.py . The python script will extract the zipfile .requirements.zip to the /tmp when the Lambda execution context is created.
Thank you for this wonderful thread.
How can I deposit uploaded image files on the server?
For example, in OnRender example, the uploaded image file is not stored on the server.
However, it would be better to save the user’s input images to the server, and possibly update the model in the future with larger image database.
Anybody have some ideas how to stack the uploaded files?
@wcneill forking this repo is probably the easiest way to get started, and if you want to create a web app that doesn’t require uploading photos you can also fork this repo I made that works in conjunction with the render template I posted above.