Is zeit free for our size of apps?
1 Like
I’m bookmarking this 
Yah, agree that this is not the place to argue about it and I really appreciate all your help to folks here on using Zeit.
Speaking of serverless approaches, is there a write-up in the docs (or elsewhere) that demonstrates how you would deploy a lambda to utilizes a fastai/pytorch model to make a prediction and return the results as JSON?
I’m also interested if there is a way to persist uploaded images, for example, in a serverless environment? For example, lets say I want folks to upload an image, return a prediction, save the image and then allow the end user to let the app know if it got the prediction right or not.
Actually I tried to that.
It sometimes works.
The problem is Lambda has a limitation of 250MB. (This is on AWS, but other cloud services also do the same)
When we add the model and all the deps, it’s pretty hard to set it to 250MB. (and 50MB zipped)
Here’s our serverless python example: https://github.com/zeit/now-examples/tree/master/python
Give it a try.
But with our current example app, we download the model when the server loads. This is problematic for serverless as it loads a new server for each and every requests.
It’ll add some latency.
1 Like
Yah that is going to be an issue unless you guys can create something like an S3 bucket where we can upload and use our models from (that would save 82MB right there).
1 Like
I think a useful PR would be to support S3 Boto storage objects etc in most places we currently take a path, such as when loading/saving a model. That way we can use object storage with stuff like sagemaker, lambda, etc.
4 Likes
Hi all.
I extended a little the Zeit blueprint described here (I did not fork it as I could not find the original repo).
I tried do make it a little more general in order to avoid to hard code in the server.py file the informations on the model. My repo with the implementation is https://github.com/francescogianferraripini/qzeit-vision. It works on the zeit paid plans
Therefore I created a model metadata json like this one (the file is modelDefinitionTemplate.json):
[ { "name":"isDamaged", "classes":["00-damage", "01-whole"], "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data1a-frozen10epochs.pth", "modelFileName":"data1a-frozen10epochs", "modelType":"resnet50", "imageSize":299 }, { "name":"damageLocation", "classes":["00-front", "01-rear", "02-side"], "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data2a-frozen10epochs-unfrozen10epochs.pth", "modelFileName":"data2a-frozen10epochs-unfrozen10epochs", "modelType":"resnet50", "imageSize":299 }, { "name":"damageSeverity", "classes":["01-minor", "02-moderate", "03-severe"], "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data3a-frozen10epochs-unfrozen10epochs.pth", "modelFileName":"data3a-frozen10epochs-unfrozen10epochs", "modelType":"resnet50", "imageSize":299 } ]
When server.py starts for the first time, it looks for a file called modelDefinition.json in the root, parses it and downloads the relevant .pth files, like the original one.
When an image is passed to the web service, it evaluates that against the many models defined in the file.
In this way we can manage and maintain a unique server.py that is independent on the models.
Moreover, I changed a little the dockerfile. It now expects a parameter, which is the url where to download modelDefinition.json, that is passed at deployment time. In this way, we can also maintain a single dockerfile and leverage the docker layers in a more simple way.
Therefore, in order to deploy it on zeit, you have now to type:
now --build-env MODEL_URL="https://raw.githubusercontent.com/francescogianferraripini/qcarcrash/master/qcarcrashModelDefinition.json"
MODEL_URL is where the modelDefinition.json can be downloaded.
I believe this approach can be useful to separate, training, developement of the serving code, and deployment/production operations and be the base for a model repository and deployment portal.
There are many todos left:
- Extend the model definition file in order to include more metadata, either useful for the prediction (like the transformations or the normalization) or for documentation about the training part, accuracy etc, and expose it also as a web service
- Generate automatically and publish on a repo the modelDefinition and the pth files from learners in the training notebook
- Create similar server.py for other scenarios like tabular.
Let me know your thoughts!
7 Likes
Hi @arunoda , I wanna try deploying on Zeit. I’m really a begineer on this kind of task and I’m using my windows 10 laptop.
I opened my windows cmd(command line prompt and when I type the first line of the tutorial “sudo apt install npm” I get this error: “sudo is not a recognized command…”.
What do recommend ? thanks
1 Like
@arunoda I also tried ubuntu for windows 10 that I have installed on my laptop to connect to Google cloud.
After the first line of the tutorial “sudo apt install npm” I get this message:
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/n/nodejs/nodejs_8.10.0~dfsg-2ubuntu0.3_amd64.deb 404 Not Found [IP: 2001:67c:1360:8001::21 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/n/nodejs/nodejs-doc_8.10.0~dfsg-2ubuntu0.3_all.deb 404 Not Found [IP: 2001:67c:1360:8001::21 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/n/nodejs/nodejs-dev_8.10.0~dfsg-2ubuntu0.3_amd64.deb 404 Not Found [IP: 2001:67c:1360:8001::21 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Then I tried the sugested command “apt-get update or try with --fix-missing”, but got another error:
Reading package lists… Done
E: Could not open lock file /var/lib/apt/lists/lock - open (13: Permission denied)
E: Unable to lock directory /var/lib/apt/lists/
W: Problem unlinking the file /var/cache/apt/pkgcache.bin - RemoveCaches (13: Permission denied)
W: Problem unlinking the file /var/cache/apt/srcpkgcache.bin - RemoveCaches (13: Permission denied)
Then I tried to go ahead and ran “sudo npm install -g now”, but I got:
sudo: npm: command not found
I would appreciate any help here.
Thanks
1 Like
You don’t need to node or npm. Simply download our Desktop app by visiting https://zeit.co/download
1 Like
Have anyone tried deploying a NLP/Text Classification Model to Zeit? If yes, can you share the code to instantiate the dummy databunch for a text classifier?
1 Like
Hey…! I wanna use fastai model for prediction in a flask web app integrated with dialogflow. Any idea how to deploy it in Zeit ??
1 Like
It appears that the Zeit tarball linked to in the deployment tutorial on the website isn’t compatible with the most recent version of fastai . I tried modifying it to use ImageDataBunch.load_empty() but ran into some trouble.
It did deploy but it appears to not actually return the prediction. I hit Analyze , the button switches to Analyzing... and that’s it. Here’s my code, and I’d be super grateful if someone wants to look through it and try and pinpoint why this might be happening. Look at lines 14, 15, 33, 34, 35 for the changes.
P.S. I’ve never actually created a web app before so I might just be missing something silly.
I’m having problems too with the new fastai code using ImageDataBunch.load_empty(). I’m getting a “TypeError: expected str, bytes or os.PathLike object, not list”
You have to use ImageDataBunch.single_from_classes
It doesn’t work anymore. It’s been replaced by ImageDataBunch.load_empty
I still see single_from_classes in the source code… is it broken?
Many apologies - we accidentally removed single_from_classes. It’s back now. Please update to the latest package.
We do plan to replace it with load_empty - so if you try that and have problems, please let @Sylvain know.
4 Likes
Thanks Jeremy for bringing it back. I was actually working on it. Seeing it going away all of sudden was a nightmare…
I did, thanks