I think a useful PR would be to support S3 Boto storage objects etc in most places we currently take a path, such as when loading/saving a model. That way we can use object storage with stuff like sagemaker, lambda, etc.
4 Likes
Hi all.
I extended a little the Zeit blueprint described here (I did not fork it as I could not find the original repo).
I tried do make it a little more general in order to avoid to hard code in the server.py file the informations on the model. My repo with the implementation is https://github.com/francescogianferraripini/qzeit-vision. It works on the zeit paid plans
Therefore I created a model metadata json like this one (the file is modelDefinitionTemplate.json):
[ { "name":"isDamaged", "classes":["00-damage", "01-whole"], "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data1a-frozen10epochs.pth", "modelFileName":"data1a-frozen10epochs", "modelType":"resnet50", "imageSize":299 }, { "name":"damageLocation", "classes":["00-front", "01-rear", "02-side"], "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data2a-frozen10epochs-unfrozen10epochs.pth", "modelFileName":"data2a-frozen10epochs-unfrozen10epochs", "modelType":"resnet50", "imageSize":299 }, { "name":"damageSeverity", "classes":["01-minor", "02-moderate", "03-severe"], "modelUrl":"https://github.com/francescogianferraripini/qcarcrash/raw/master/data3a-frozen10epochs-unfrozen10epochs.pth", "modelFileName":"data3a-frozen10epochs-unfrozen10epochs", "modelType":"resnet50", "imageSize":299 } ]
When server.py starts for the first time, it looks for a file called modelDefinition.json in the root, parses it and downloads the relevant .pth files, like the original one.
When an image is passed to the web service, it evaluates that against the many models defined in the file.
In this way we can manage and maintain a unique server.py that is independent on the models.
Moreover, I changed a little the dockerfile. It now expects a parameter, which is the url where to download modelDefinition.json, that is passed at deployment time. In this way, we can also maintain a single dockerfile and leverage the docker layers in a more simple way.
Therefore, in order to deploy it on zeit, you have now to type:
now --build-env MODEL_URL="https://raw.githubusercontent.com/francescogianferraripini/qcarcrash/master/qcarcrashModelDefinition.json"
MODEL_URL is where the modelDefinition.json can be downloaded.
I believe this approach can be useful to separate, training, developement of the serving code, and deployment/production operations and be the base for a model repository and deployment portal.
There are many todos left:
- Extend the model definition file in order to include more metadata, either useful for the prediction (like the transformations or the normalization) or for documentation about the training part, accuracy etc, and expose it also as a web service
- Generate automatically and publish on a repo the modelDefinition and the pth files from learners in the training notebook
- Create similar server.py for other scenarios like tabular.
Let me know your thoughts!
7 Likes
Hi @arunoda , I wanna try deploying on Zeit. I’m really a begineer on this kind of task and I’m using my windows 10 laptop.
I opened my windows cmd(command line prompt and when I type the first line of the tutorial “sudo apt install npm” I get this error: “sudo is not a recognized command…”.
What do recommend ? thanks
1 Like
@arunoda I also tried ubuntu for windows 10 that I have installed on my laptop to connect to Google cloud.
After the first line of the tutorial “sudo apt install npm” I get this message:
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/n/nodejs/nodejs_8.10.0~dfsg-2ubuntu0.3_amd64.deb 404 Not Found [IP: 2001:67c:1360:8001::21 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/n/nodejs/nodejs-doc_8.10.0~dfsg-2ubuntu0.3_all.deb 404 Not Found [IP: 2001:67c:1360:8001::21 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/n/nodejs/nodejs-dev_8.10.0~dfsg-2ubuntu0.3_amd64.deb 404 Not Found [IP: 2001:67c:1360:8001::21 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
Then I tried the sugested command “apt-get update or try with --fix-missing”, but got another error:
Reading package lists… Done
E: Could not open lock file /var/lib/apt/lists/lock - open (13: Permission denied)
E: Unable to lock directory /var/lib/apt/lists/
W: Problem unlinking the file /var/cache/apt/pkgcache.bin - RemoveCaches (13: Permission denied)
W: Problem unlinking the file /var/cache/apt/srcpkgcache.bin - RemoveCaches (13: Permission denied)
Then I tried to go ahead and ran “sudo npm install -g now”, but I got:
sudo: npm: command not found
I would appreciate any help here.
Thanks
1 Like
You don’t need to node or npm. Simply download our Desktop app by visiting https://zeit.co/download
1 Like
Have anyone tried deploying a NLP/Text Classification Model to Zeit? If yes, can you share the code to instantiate the dummy databunch for a text classifier?
1 Like
Hey…! I wanna use fastai model for prediction in a flask web app integrated with dialogflow. Any idea how to deploy it in Zeit ??
1 Like
It appears that the Zeit tarball linked to in the deployment tutorial on the website isn’t compatible with the most recent version of fastai . I tried modifying it to use ImageDataBunch.load_empty() but ran into some trouble.
It did deploy but it appears to not actually return the prediction. I hit Analyze , the button switches to Analyzing... and that’s it. Here’s my code, and I’d be super grateful if someone wants to look through it and try and pinpoint why this might be happening. Look at lines 14, 15, 33, 34, 35 for the changes.
P.S. I’ve never actually created a web app before so I might just be missing something silly.
I’m having problems too with the new fastai code using ImageDataBunch.load_empty(). I’m getting a “TypeError: expected str, bytes or os.PathLike object, not list”
You have to use ImageDataBunch.single_from_classes
It doesn’t work anymore. It’s been replaced by ImageDataBunch.load_empty
I still see single_from_classes in the source code… is it broken?
Many apologies - we accidentally removed single_from_classes. It’s back now. Please update to the latest package.
We do plan to replace it with load_empty - so if you try that and have problems, please let @Sylvain know.
4 Likes
Thanks Jeremy for bringing it back. I was actually working on it. Seeing it going away all of sudden was a nightmare… 
I did, thanks
How to retrain my model after creating it from a different dataset, on a new but few datasets and classes ??
For example,if I created a model with 5 classes and 50 data instances per class. Now I want it to retrain with new data and new class , say 5 data points of 6th class without training it from scratch (i.e without building a whole new databunch with all the data/classes) ??
A lot of people are struggling to deploy their flask app on Heroku because of the size and installation of a library, I have written a guide on GitHub in case anyone needs it
I am also writing a blog which will be coming soon
4 Likes
If any of you are struggling with Heroku or other providers, I’d love for you to try Render. The guide for fastai-v3 is here: https://course-v3.fast.ai/deployment_render.html
We don’t have any size restrictions on Docker images.
5 Likes
Shankar, were you able to deploy your classifier on Render?