I’ve submitted a PR with my concept of LM inference API. Let me know if you like it (or not). https://github.com/fastai/fastai/pull/1128
3 Likes
Follow this guide: https://zeit.co/docs/v2/domains-and-aliases/adding-a-domain#3-using-a-custom-domain-dns-managed-by-now
1 Like
Thanks, I did follow the guide and the error message. I added the namesserver yet I still get the same error message.
Was trying to deploy segmentation model using zeit but unable to do so.
Step 7/9 : RUN python app/server.py
—> Running in 57b7b5612ce1
Traceback (most recent call last):
File “app/server.py”, line 37, in
learn = loop.run_until_complete(asyncio.gather(*tasks))[0]
File “/usr/local/lib/python3.6/asyncio/base_events.py”, line 473, in run_until_complete
return future.result()
File “app/server.py”, line 32, in setup_learner
learn.load(model_file_name)
File “/usr/local/lib/python3.6/site-packages/fastai/basic_train.py”, line 204, in load
self.model.load_state_dict(torch.load(self.path/self.model_dir/f’{name}.pth’, map_location=device))
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 358, in load
return _load(f, map_location, pickle_module)
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 519, in _load
magic_number = pickle_module.load(f)
_pickle.UnpicklingError: invalid load key, ‘<’.
The command ‘/bin/sh -c python app/server.py’ returned a non-zero code: 1
On running in zeid folder on Ubunto terminal and similar issue when running on paperspace machine.
sudo npm install -g now
I get the following error:
139 error Linux 4.4.0-17134-Microsoft 140 error argv “/usr/bin/nodejs” “/usr/bin/npm” “install” “-g” “now” 141 error node v4.2.6 142 error npm v3.5.2 143 error file sh 144 error code ELIFECYCLE 145 error errno ENOENT 146 error syscall spawn 147 error now@12.1.2 postinstall:
node download/install.js147 error spawn ENOENT 148 error Failed at the now@12.1.2 postinstall script ‘node download/install.js’. 148 error Make sure you have the latest version of node.js and npm installed. 148 error If you do, this is most likely a problem with the now package, 148 error not with npm itself. 148 error Tell the author that this fails on your system: 148 error node download/install.js 148 error You can get information on how to open an issue for this project with: 148 error npm bugs now 148 error Or if that isn’t available, you can get their info via: 148 error npm owner ls now 148 error There is likely additional logging output above.
npm -v 6.4.1
node -v 11.1.0@vikashkmr: the download URL to your model is not correct
Make sure you do this step properly
Note: the download link is the one which starts the file download directly—and is normally different than the share link which presents you with a view to download the file (use https://rawdownload.now.sh/ if needed)
Refer:
https://forums.fast.ai/t/new-guide-for-easy-web-app-deployment/29616/15
Here is my little web app, thanks for everyone who helped.
It’s a little web app classifying type of a container.
https://jianyang.ml/
Hi @simonw!
Thanks a lot for sharing this! I was able to create a MVP web app in no time thanks to this.
I have a question. Is this actually a full REST Api? Is it possible to call the API via curl?
The command curl -X POST -F image=@teddy-bear.jpg 'http://localhost/upload' gave me an Internal Server Error as response. I am asking this because I want to use your boilerplate to create a backend that could be called from an Android client.
By the way I slightly modified your code to work with the latest fastai version. Check it out here. The main issue was to use create_cnn instead of ConvLearner.
Thanks a lot for reading this! 
@Andreas_Daiminger check my server boilerplate example https://github.com/IaroslavR/fastai-rest-server-template
1 Like
@elruso Thanks for sharing this. Looks cool. 
I built a REST API with Flask in the meantime and dockerized it. All based on suggestions from the fast.ai forum. Check it out here
I created a repository to test PyTorch models in Flask and then deploy them to AWS Lambda using Zappa.
Link to repository: https://github.com/pedrohbtp/pytorch-zappa-serverless
The Gif from above works from an AWS Lambda deployment of the pre-trained english Language Model available in Fastai. You can test it out here: https://pedrohbtp.github.io/language-model/
AWS Lambda has some size limitations, so I had to use fastai’s PyTorch models instead of installing the entire library.
If someone wants an introduction to Flask, I wrote a little about it here.
8 Likes
@Andreas_Daiminger pay attention to paragraph three of https://www.codacy.com/blog/five-ways-to-slim-your-docker-images/ this way you can minimize your image simple way
1 Like
It appears that the Zeit tarball linked to in the deployment tutorial on the website isn’t compatible with the most recent version of fastai. I tried modifying it to use ImageDataBunch.load_empty() but ran into some trouble.
It did deploy but it appears to not actually return the prediction. I hit Analyze, the button switches to Analyzing... and that’s it. Here’s my code, and I’d be super grateful if someone wants to look through it and try and pinpoint why this might be happening. Look at lines 14, 15, 33, 34, 35 for the changes.
P.S. I’ve never actually created a web app before so I might just be missing something silly.
2 Likes
Thanks for your link.
I was able to train my model and get something up and running more or less along your guidelines.
I had to do a lot to get everything to fit inside of lambda as the default deployment was way way too big. So much too big it makes me think you perhaps left out some of that in your repo?
I couldn’t find anyway to tree-shake in python “automatically” so I did myself a bit and was able to shrink enough.
I also had to load my model into memory directly from s3 rather than storing it locally as I couldn’t fit both the code and model in the scratch space.
My ‘deployed’ version is here: https://gdoteof.github.io/neuralnet_stuff/
I have no done a writeup (though all the code, complete with my fumblings is in the above repo).
If anyone ends up on this thread trying to figure out how to shrink their deployment to fit on lambda, take a look at my zappa config to see some things you can remove.
I am glad you also made it work. Great work there.
If there is interest I can try to explain a little better the deployment and create more documentation on the readme.
Indeed there is no auto tree-shaking. When I did it, I had to choose which of the libraries I would include. That is why I didn’t even include the actual fastai, but only Pytorch related dependencies. I apologize if there was any trouble. I am not sure if I left any unnecessary dependencies there. I can double check.
It is really not possible to fit the model and the code into the lambda space. We have to read it from s3 every time which might make loading a little bit slower.
Did you remove code by actually deleting the libraries rather than just excluding them from the payload? That may be why I missed it.
I would encourage you to expand on what you’ve written already. I found it very useful, but still struggled (a lot) with actually getting it deployed. Partly due to having a slow uplink at the time I was working on (250MB uploads on 1Mb/s make for slow iterations).
Particularly if you could expand on my list of excludes I think that might be helpful. And in general, just clean up your repo a bit to make it ‘accurate’ for deploying on lambda. Your latest commit for example is attempting to use scratch space rather than load into memory directly.
I actually spent several hours trying to make that work because I thought you were able to 
Glad to know you ended up loading from s3 directly as well.
Thank you for the feedback!
I did not really remove the libraries manually
I will update the repository
Does anyone run successfully fast.ai on embedded system like a raspberry pi ? Is it enough power for deep learning or can you suggest me alternatives ?
Thank you in advances
2 Likes