Hosting a simple web application

i’m trying to do this same exact thing, any luck?

It might be connected with your local host env. If you run something on heroku, it has different parameters. First of all, it might not have gpu.
So your model should run on cpu. If there is an option to save fast.ai model for production - it should be the way.

If you don’t find the way to save model for inference (== production), hacky way would be installing fast.ai on heroku.

I’m also interested, is it an option to run model without gpu, so let’s crack this topic %)

by the way, the link on github - his demo didn’t work for me.

Hey guys, I have managed to host in on Google Cloud Engine, albeit some hacky approach. I will write a blog on my experience when I have time soon.

As for the heroku error above, it is due to the torch package not available in pip. To fix it, replace torch with in the requirements.txt: http://download.pytorch.org/whl/cpu/torch-0.3.1-cp36-cp36m-linux_x86_64.whl
But heroku won’t be able to run the web app after I deploy it as the RAM is just 512MB on free tier. The inference model forward pass on CPU takes much more memory than that.

3 Likes

It will be very helpful if anyone that has succeeded to install and run fastai env in production could share with us his experience since i am trying to build a small web api to do predictions using fastai library

can you link to your google cloud implementation in the meantime? would appreciate the blog post also!! :slight_smile:

hello @jakcycsl, can you provide more info of how you build your app? :slight_smile:

Hi jakcycsi,

Do you mind sharing the code you used for building the web app?

Hi everyone, just came across this thread. ICYMI, I posted a step-by-step overview (and github repo) of creating and deploying a web app demo of a fastai-trained model with flask, gunicorn, nginx, and remote hosting:

7 Likes

Oh lol! So am not just along stuck in this haha, did you get any solution or rather did luck favor?

With Regards,
Xovo Larjem https://putlocker.ooo/
https://viamichelin.onl/
https://googleearth.onl/

In my own experience, tl;dr; most of my recent production deployment successes are thanks to Docker container, simplifying the deployment process and reduce the frictions.

For example, on Heroku, you can customize your stack and deploy using container by creating and building your own image up to your runtime environment specs you need like PyTorch version 0.3.x, Python 3.6, etc if your model is optimized to run on CPU. It’s hard but possible to make this works on the free hobby plan. That’s for the model service (Flask). As for the front-end web (interface) and web back-end (I use Django model/ORM and REST Framework for API to save front-end data to PostgreSQL DB). Both of this can be easily fit and run well on Heroku for free hobby project.

Yeah, highly do check out Dave’s post. I get a lot done with his idea and it took me far enough until our traffic volume out scale this architecture. For non-serious / hobby project with free or low cost plan, I use Heroku :smiley:

1 Like

A lot of people are struggling to deploy their app on heroku, I have written a detailed guide on github in case anyone needs it.

I am also writing a blog which will be coming soon :slight_smile:

4 Likes

Great! Looking forward to working through your example. Just tried to deploy an NLP app to Heroku and discovered that there are issues including a Spacy model. Apparently this would require an expensive paid subscription? Going to try on Paperspace.

Hi i have no knowledge about how to deploy your model in a web application. I would be highly grateful if someone can help me with this. I am using google colab.

Has anyone overcome the slug limits on Heroku? After lesson 1, trained a model, downloaded the trained model to my laptop, served the model for inference on my local host and it runs great. I have deployed in Heroku before, but per other dialog in this thread the Heroku slug size max is 500M, and my deployment slug size “Compiled slug size: 824.6M is too large (max is 500M)”. I see some references to the cpu version of pytorch being smaller, but not sure how to use that with fastai. Appreciate any thoughts, or recommendations on other free or inexpensive approaches different than Heroku. I am more using to prove out end to end, so performance is not critical.

I’m in the same spot and came here to look for some input. Works like a charm on localhost, but didn’t see the Heroku slug size limitation coming.

My only hints so far are that, as you mentioned, PyTorch has the CPU version which you can find a .whl for on their website and it’s about 68MB. However the instructions to install the wheel are followed by a “pip3 install torchvision” command, and through doing that, it starts collecting the normal CUDA torch which is about 580MB alone and the main culprit here. Another issue is that installing fastai starts this same process of collecting torch etc. There are some instructions in the docs under installation where you can control which dependencies get installed, however while the pieces of the puzzle seem to be here, I’m overwhelmed at this point trying to navigate all this through Heroku. I’ll update if I figure it out.

EDIT: Getting it to work on Heroku:

Surprise surprise, the docs had the simple answer. Installing torchvision separately etc. wasn’t necessary. One thing though, the wheel there is out of date, although it might still work, I didn’t test it.

What I did to get it working for myself was add the correct wheel for pytorch(https://pytorch.org/get-started/locally/ , Stable, Linux, Pip, Python 3.6, Cuda: None) and fastai to requirements.txt.

Make sure you don’t have separate lines for torch, torchvision, I also removed my old fastai line which used to read fastai==1.0.51. I left the other dependencies in place for starters and it ended up working out.

Add these lines to requirements.txt:

https://download.pytorch.org/whl/cpu/torch-1.0.1.post2-cp36-cp36m-linux_x86_64.whl
fastai

With my other libraries, that left my slug at roughly 340MB. That apparently isn’t great, and there are ways to reduce that to improve performance, or so I hear, but that will get you up and running without hitting the 500MB roof.

3 Likes

This doesn’t work for me and I’m clueless. Thus slug size still crosses well beyond 600MB.
I also opened a Github issue here :frowning:

EDIT! Put up what worked for me on the Github post.

Not attached in any form to this book from Manning but “AI as a service” is just what most of us are looking for and based upon AWS, so worth a link here maybe? https://www.manning.com/books/ai-as-a-service . We get a 42% discount with the coupon provided by the author’s intro here https://freecontent.manning.com/slideshare-serverless-ai-solutions/ . Good luck.

Hey everyone, I have a service running on GCP running inference. I’m working on building a model to detect when a answering machine ‘beep’ is detected during a call.
Its deployed on GCP but its really slow during inference.
But at least, ya’ll will be able to see how its built to be deployed on GCP.

that solved it for me, thanks :slight_smile:

it worked for me. thank you so much :smiley:

updated: we should use the newer version of torch as following
https://download.pytorch.org/whl/cpu/torch-1.1.0-cp36-cp36m-linux_x86_64.whl