# New guide for easy web app deployment

Thanks to @navjots, we now have a guide showing how to easily deploy your fastai model in production using Zeit:

https://course-v3.fast.ai/deployment_zeit.html

It includes a minimal web app to get you started - here it is!:

https://fastai-teddies.now.sh/

Guides for other platforms most welcome. If you create one, please try to automate as much as possible - as you see in this Zeit example.

33 Likes

it’s a great app and the guide is great too. much appreciated!

it’s not a criticism. just a question on image classifiers in general. say to classify cats vs dogs, that’s great accuracy is fantastic. however if you upload picture of a horse for example, it will not tell you that the picture is ‘neither cat or dog’, it will rather assign one of classes ‘cat’ or ‘dog’, the one with higher probability. but clearly it’s not a cat or dog. so i am thinking on how to work with this. one way will be to give confidence in the classification, so user can judge also based on that score.
is there another way to do it?
so cats and dogs are simple problems for people to solve and this problem may seem not relevant. however if the classification was on things that people can’t tell the difference and were relying on DL to make decision, like city pictures from satellite, or some medical condition, that would be more relevant, no?

3 Likes

Yes you can also train an “other” category. And if the confidence isn’t high, you could also report that as “other” (or similar).

5 Likes

You could also do first training with 2 classes of interest (say, cats and dogs), and once you have your classifier ready, take a validation set of cats and dogs, and add to it images of other things. Then you will have to tune 2 thresholds t0<0.5< t1. If cat=0 and dog=1, then you output cat if the estimated probability is < t0, output dog if the estimated probability is >t2, and you output “other” otherwise. The hyper-parameters t1, t2 can be tuned to optimize a meaningful metric on the validation set. I

6 Likes

Probably it is also worth to cover `git lfs` functionality for someone who would like to use Git to store large files.

2 Likes

i wonder if the technique used in lesson 9 DL2 SSD where if it is not one of the 20 classes it defaults to background could be applied in this case. so if it’s not dogs and not cats it defaults to others.

Yes - that’s pretty similar to what @martinmm mentioned. The key would be to not use softmax, but use the scores directly.

4 Likes

Has anyone experimented with DVC for his?

3 Likes

Have never seen this thing previously but sounds interesting. Definitely worth to try. Is it something built on top of Git? Looks like a Python package specifically tailored to help with ML-related types of files processing.

Can we run opencv inside this web app created using now services and zeit?

Hi all.

@Jeremy , I use another way to deploy my guitar classifier app for cheap. I’m using Dokku on a VPC instance. This gives you one to many always-on apps for about 3\$ a month…

Blog post is here:

8 Likes

I followed the guide, but the app’s not working for me. The build fails. Followed the guide exactly as it was written. I used the download link for dropbox and changed the classes in the server.py file. Can someone help me out? Something to do with “invalid load key”.

Building…
Sending build context to Docker daemon 36.86kB
—> ffafb5882b66
Step 2/9 : RUN apt update
—> Using cache
—> 818132902904
Step 3/9 : RUN apt install -y python3-dev gcc
—> Using cache
—> 1f3b479f51ea
Step 4/9 : ADD requirements.txt requirements.txt
—> Using cache
—> Running in 14b0e07be2c2
Traceback (most recent call last):
File “app/server.py”, line 37, in
File “/usr/local/lib/python3.6/asyncio/base_events.py”, line 473, in run_until_complete
return future.result()
File “app/server.py”, line 32, in setup_learner
File “/usr/local/lib/python3.6/site-packages/fastai/basic_train.py”, line 204, in load
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 358, in load
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 519, in _load
Error! Build failed

1 Like

From their site:

“DVC runs on top of any Git repository and is compatible with any standard Git server or provider (Github, Gitlab, etc). Data file contents can be shared by network-accessible storage or any supported cloud solution. DVC offers all the advantages of a distributed version control system — lock-free, local branching, and versioning.”

“Use S3, Azure, GCP, SSH, SFTP, rsync or any network-attached storage to store data. The list of supported protocols is constantly expanding.”

I will definitely check it out when I find the time…

I’m pretty sure that means your model URL is incorrect. Try using it locally with `wget` and then open the file that gets downloaded to check it’s correct.

4 Likes

Did you do this step @Mauro ?

If you used dropbox, the sharelink normally ends with `?dl=0`
You ll have to change that to `?raw=1`

1 Like

1 Like

I’m currently running into this exact pickling issue with the same invalid load key (<).

The dropbox url for the model has been converted to the raw download version. Any advice for debugging this?

Edit: there was already a model file in the app/models directory. I needed to delete it for things to work.

Although a new issue has arisen. I’m trying to deploy a super resolution model that returns an image as an output. The standard Zeit template is designed to return a `JSONResponse`. If my final output is either a torch tensor or a fastai Image object, what is the best way to return that?

1 Like

@sgugger @lesscomfortable Help
I am trying to deploy my app on zeit, but it continues to give me errors. I am using the same code that is provided in the deployment guide.

If any of you are struggling with cloud providers for deployment, I’d love for you to try Render. The guide for fastai-v3 is here: https://course-v3.fast.ai/deployment_render.html

We don’t have any size restrictions on Docker images, and I’m around to answer questions and help with debugging. (I’m the founder/CEO of Render and previously built Crestle).

9 Likes