Thanks to @navjots, we now have a guide showing how to easily deploy your fastai model in production using Zeit:
https://course-v3.fast.ai/deployment_zeit.html
It includes a minimal web app to get you started - here it is!:
https://fastai-teddies.now.sh/
Guides for other platforms most welcome. If you create one, please try to automate as much as possible - as you see in this Zeit example.
it’s a great app and the guide is great too. much appreciated!
it’s not a criticism. just a question on image classifiers in general. say to classify cats vs dogs, that’s great accuracy is fantastic. however if you upload picture of a horse for example, it will not tell you that the picture is ‘neither cat or dog’, it will rather assign one of classes ‘cat’ or ‘dog’, the one with higher probability. but clearly it’s not a cat or dog. so i am thinking on how to work with this. one way will be to give confidence in the classification, so user can judge also based on that score.
is there another way to do it?
so cats and dogs are simple problems for people to solve and this problem may seem not relevant. however if the classification was on things that people can’t tell the difference and were relying on DL to make decision, like city pictures from satellite, or some medical condition, that would be more relevant, no?
what’s your thoughts?
Yes you can also train an “other” category. And if the confidence isn’t high, you could also report that as “other” (or similar).
You could also do first training with 2 classes of interest (say, cats and dogs), and once you have your classifier ready, take a validation set of cats and dogs, and add to it images of other things. Then you will have to tune 2 thresholds t0<0.5< t1. If cat=0 and dog=1, then you output cat if the estimated probability is < t0, output dog if the estimated probability is >t2, and you output “other” otherwise. The hyper-parameters t1, t2 can be tuned to optimize a meaningful metric on the validation set. I
Probably it is also worth to cover git lfs functionality for someone who would like to use Git to store large files.
i wonder if the technique used in lesson 9 DL2 SSD where if it is not one of the 20 classes it defaults to background could be applied in this case. so if it’s not dogs and not cats it defaults to others.
Yes - that’s pretty similar to what @martinmm mentioned. The key would be to not use softmax, but use the scores directly.
Have never seen this thing previously but sounds interesting. Definitely worth to try. Is it something built on top of Git? Looks like a Python package specifically tailored to help with ML-related types of files processing.
Can we run opencv inside this web app created using now services and zeit?
Hi all.
@Jeremy , I use another way to deploy my guitar classifier app for cheap. I’m using Dokku on a VPC instance. This gives you one to many always-on apps for about 3$ a month…
Blog post is here:
I followed the guide, but the app’s not working for me. The build fails. Followed the guide exactly as it was written. I used the download link for dropbox and changed the classes in the server.py file. Can someone help me out? Something to do with “invalid load key”.
Building…
Sending build context to Docker daemon 36.86kB
—> ffafb5882b66
Step 2/9 : RUN apt update
—> Using cache
—> 818132902904
Step 3/9 : RUN apt install -y python3-dev gcc
—> Using cache
—> 1f3b479f51ea
Step 4/9 : ADD requirements.txt requirements.txt
—> Using cache
—> Running in 14b0e07be2c2
Traceback (most recent call last):
File “app/server.py”, line 37, in
learn = loop.run_until_complete(asyncio.gather(*tasks))[0]
File “/usr/local/lib/python3.6/asyncio/base_events.py”, line 473, in run_until_complete
return future.result()
File “app/server.py”, line 32, in setup_learner
learn.load(model_file_name)
File “/usr/local/lib/python3.6/site-packages/fastai/basic_train.py”, line 204, in load
self.model.load_state_dict(torch.load(self.path/self.model_dir/f’{name}.pth’, map_location=device))
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 358, in load
return _load(f, map_location, pickle_module)
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 519, in _load
magic_number = pickle_module.load(f)
_pickle.UnpicklingError: invalid load key, ‘<’.
Error! Build failed
From their site:
“DVC runs on top of any Git repository and is compatible with any standard Git server or provider (Github, Gitlab, etc). Data file contents can be shared by network-accessible storage or any supported cloud solution. DVC offers all the advantages of a distributed version control system — lock-free, local branching, and versioning.”
…
“Use S3, Azure, GCP, SSH, SFTP, rsync or any network-attached storage to store data. The list of supported protocols is constantly expanding.”
I will definitely check it out when I find the time…
I’m pretty sure that means your model URL is incorrect. Try using it locally with wget and then open the file that gets downloaded to check it’s correct.
Note: the download link is the one which starts the file download directly—and is normally different than the share link which presents you with a view to download the file (use https://rawdownload.now.sh/ if needed)
Did you do this step @Mauro ?
If you used dropbox, the sharelink normally ends with ?dl=0
You ll have to change that to ?raw=1
(or use https://rawdownload.now.sh/)
I used the download link itself. Not the share link. It’s different than the link that one gets from rawdownload.now.sh. But it’s working fine for me now using that site to convert the share link. Thanks.
It was the url. I should have used https://rawdownload.now.sh/. I was using the download link explicitly.
I’m currently running into this exact pickling issue with the same invalid load key (<).
The dropbox url for the model has been converted to the raw download version. Any advice for debugging this?
Edit: there was already a model file in the app/models directory. I needed to delete it for things to work.
Although a new issue has arisen. I’m trying to deploy a super resolution model that returns an image as an output. The standard Zeit template is designed to return a JSONResponse. If my final output is either a torch tensor or a fastai Image object, what is the best way to return that?
@sgugger @lesscomfortable Help
I am trying to deploy my app on zeit, but it continues to give me errors. I am using the same code that is provided in the deployment guide.
If any of you are struggling with cloud providers for deployment, I’d love for you to try Render. The guide for fastai-v3 is here: https://course-v3.fast.ai/deployment_render.html
We don’t have any size restrictions on Docker images, and I’m around to answer questions and help with debugging. (I’m the founder/CEO of Render and previously built Crestle).
