Deployment Platform: Render ✅

gbubs · September 30, 2019, 10:09pm

Yes, I was able to do it. You can check it here. But in my case, I just used a small subset of images for testing. Therefore the predictions at the end were bad but it works and it worked on render. Remember if your export.pkl file is more than 100MB free render account won’t work. Hope that helps.

you will also notice that I didn’t use “acc_camvid” model to calculate accuracy. It seems new functions need a different way of handling to deploy models on render.

harsh_el · October 1, 2019, 11:55am

@gbubs Thanks for getting back to me. Could be able to share your server.py and other HTML and CSS scripts required for deploying the model on render? It would be a great help if you could.

Thanks

gbubs · October 1, 2019, 6:32pm

You can fork my Github repo

harsh_el · October 2, 2019, 10:07am

@gbubs Hey, thank you for your response. I wanted to ask the type of output that you are getting while running the model in the browser. Are you getting a segmentation image as the output or a string output?
Because the original script provided by fastai is equipped to show the output label in string format but since the output here we are expecting is a segmentation map of the input image, so have you made the required changes in the original script to display the segmentation map as the output?
Also if it isn’t too much, can you start your render server so that I can check how your model is working in the browser.

Thanks

mrfabulous1 · October 2, 2019, 11:42am

Hi gbubs hope your well!
Thank you for taking the time to share your repo and for the time you spent making it.
Cheers mrfabulous1

gbubs · October 2, 2019, 6:47pm

The output is the segmentation map. Unfortunately render is a paid service after few days of use. So I had to discontinue using it.

harsh_el · October 3, 2019, 7:02am

@gbubs So after cloning your repo I deployed the model on render. As you said, I am not able to get the segmentation map as the output. Once you press the analyze button what is the output that you are getting? In order to get an image as an output as opposed to a string, you would have made some changes in the original script right?
I would really appreciate your help if you can point me the right direction as my main aim is to get the segmentation image as the output once I press the analyze button.

Thanks

gbubs · October 3, 2019, 5:11pm

Camvid code output is a segmentation map (not a string) and this info should be in export.pkl. Therefore in theory Render app should give you the image like in my gist code. I couldn’t able to see the output as image as my export.pkl was more than 100 MB and therefore free Render service failed on me. If you are trying with paid Render services, you should be able to see it. That’s all I can help with you. Good luck with your project.

SisengCo · October 4, 2019, 6:18am

Hello @anurag
I have ended my web serveices yet I am being billed by render.com monthly…
there is no opt out button anywhere on my page.

I WANT TO OPT OUT PLEASE!!

anurag · October 4, 2019, 3:01pm

Just email support@render.com with the email on your account and we’ll close it. Sorry about the trouble.

davide445 · October 5, 2019, 2:05pm

Hi, trying to use locally the docker image for testing.

Uploaded on Dropbox my export.pkl file, generated the direct download (btw I think the link generated from https://syncwithtech.blogspot.com/p/direct-download-link-generator.html seems not to work, so I just used the Dropbox share link and put “?raw=1” at the end), edited the server.py with the export_file_url and classes.

Since I received some errors that from this forum I discovered being a mismatch btw torch versions I tried first to update torch (and torchvision) in my env and next align the requirements file with mine env

aiofiles==0.4.0
aiohttp==3.5.4
asyncio==3.4.3
fastai==1.0.58
https://download.pytorch.org/whl/cpu/torch-1.1.0-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/cpu/torchvision-0.3.0-cp37-cp37m-linux_x86_64.whl
numpy==1.15.4
starlette==0.12.0
uvicorn==0.7.1
python-multipart==0.0.5

Trying to recompile the docker image using

docker build -t fastai-v3 . && docker run --rm -it -p 5000:5000 fastai-v3

I got this error (standard bears app is working fine)

Step 6/8 : RUN python app/server.py
—> Running in 6230a2254f42
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.loss.CrossEntropyLoss’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.conv.Conv2d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.batchnorm.BatchNorm2d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.activation.ReLU’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torchvision.models.resnet.Bottleneck’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.pooling.AdaptiveAvgPool2d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.batchnorm.BatchNorm1d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.dropout.Dropout’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.linear.Linear’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
Removing intermediate container 6230a2254f42
—> 3276c39adff2
Step 7/8 : EXPOSE 5000
—> Running in eb462329fca0
Removing intermediate container eb462329fca0
—> 2fe52c97d2bf
Step 8/8 : CMD [“python”, “app/server.py”, “serve”]
—> Running in 8daa88c2b2f2
Removing intermediate container 8daa88c2b2f2
—> 21786884c221
Successfully built 21786884c221
Successfully tagged fastai-v3:latest
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
ModuleNotFoundError: No module named ‘numpy.core._multiarray_umath’
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.loss.CrossEntropyLoss’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.conv.Conv2d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.batchnorm.BatchNorm2d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.activation.ReLU’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torchvision.models.resnet.Bottleneck’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.pooling.AdaptiveAvgPool2d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.batchnorm.BatchNorm1d’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.dropout.Dropout’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
/usr/local/lib/python3.7/site-packages/torch/serialization.py:454: SourceChangeWarning: source code of class ‘torch.nn.modules.linear.Linear’ has changed. you can retrieve the original source code by accessing the object’s source attribute or set torch.nn.Module.dump_patches = True and use the patch tool to revert the changes.
warnings.warn(msg, SourceChangeWarning)
INFO: Started server process [1]
INFO: Waiting for application startup.
INFO: Uvicorn running on http://0.0.0.0:5000 (Press CTRL+C to quit)

Now i have this problem trying to run it locally and another problem on my notebook where

learn = create_cnn(data, models.resnet50, metrics=error_rate)
UserWarning: create_cnn is deprecated and is now named cnn_learner. warn("create_cnn is deprecated and is now named cnn_learner.")

but if I use

learn = cnn_learner(data, models.resnet50, metrics=error_rate)
NameError: name ‘cnn_learner’ is not defined

ApacheT101 · October 5, 2019, 3:43pm

I am trying to develop a phone classifier model for my university project. I’ve already trained my model and I encountered a problem when I am trying to deploy the model by executing python app/server.py serve. I read a post(Unexpected key(s) in state_dict: "model", "opt") and I suspect the problem is due to different fast.ai version running between my anaconda and Google Colab.

Therefore,I’ve tried to check the version of fastai in my computer by using pip list fastai, conda list fastai and import fastai; fastai. version in my Google colab(I used Google Colab to develop my model) but the results are the same(fastai version = 1.0.59). I even tried to update my fastai version in Google Colab but in vain. Here is the exception code:

Traceback (most recent call last):
  File "app/server.py", line 37, in <module>
    learn = loop.run_until_complete(asyncio.gather(*tasks))[0]
  File "C:\ProgramData\Anaconda3\lib\asyncio\base_events.py", line 584, in run_until_complete
    return future.result()
  File "app/server.py", line 32, in setup_learner
    learn.load(model_file_name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\fastai\basic_train.py", line 279, in load
    get_model(self.model).load_state_dict(state, strict=strict)
  File "C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 845, in load_state_dict
    self.__class__.__name__, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for Sequential:
        Missing key(s) in state_dict: "0.0.weight", "0.1.weight", "0.1.bias", "0.1.running_mean", "0.1.running_var", "0.4.0.conv1.weight", "0.4.0.bn1.weight", "0.4.0.bn1.bias", "0.4.0.bn1.running_mean", "0.4.0.bn1.running_var", "0.4.0.conv2.weight", "0.4.0.bn2.weight", "0.4.0.bn2.bias", "0.4.0.bn2.running_mean", "0.4.0.bn2.running_var", "0.4.1.conv1.weight", "0.4.1.bn1.weight", "0.4.1.bn1.bias", "0.4.1.bn1.running_mean", "0.4.1.bn1.running_var", "0.4.1.conv2.weight", "0.4.1.bn2.weight", "0.4.1.bn2.bias", "0.4.1.bn2.running_mean", "0.4.1.bn2.running_var", "0.4.2.conv1.weight", "0.4.2.bn1.weight", "0.4.2.bn1.bias", "0.4.2.bn1.running_mean", "0.4.2.bn1.running_var", "0.4.2.conv2.weight", "0.4.2.bn2.weight", "0.4.2.bn2.bias", "0.4.2.bn2.running_mean", "0.4.2.bn2.running_var", "0.5.0.conv1.weight", "0.5.0.bn1.weight", "0.5.0.bn1.bias", "0.5.0.bn1.running_mean", "0.5.0.bn1.running_var", "0.5.0.conv2.weight", "0.5.0.bn2.weight", "0.5.0.bn2.bias", "0.5.0.bn2.running_mean", "0.5.0.bn2.running_var", "0.5.0.downsample.0.weight", "0.5.0.downsample.1.weight", "0.5.0.downsample.1.bias", "0.5.0.downsample.1.running_mean", "0.5.0.downsample.1.running_var", "0.5.1.conv1.weight", "0.5.1.bn1.weight", "0.5.1.bn1.bias", "0.5.1.bn1.running_mean", "0.5.1.bn1.running_var", "0.5.1.conv2.weight", "0.5.1.bn2.weight", "0.5.1.bn2.bias", "0.5.1.bn2.running_mean", "0.5.1.bn2.running_var", "0.5.2.conv1.weight", "0.5.2.bn1.weight", "0.5.2.bn1.bias", "0.5.2.bn1.running_mean", "0.5.2.bn1.running_var", "0.5.2.conv2.weight", "0.5.2.bn2.weight", "0.5.2.bn2.bias", "0.5.2.bn2.running_mean", "0.5.2.bn2.running_var", "0.5.3.conv1.weight", "0.5.3.bn1.weight", "0.5.3.bn1.bias", "0.5.3.bn1.running_mean", "0.5.3.bn1.running_var", "0.5.3.conv2.weight", "0.5.3.bn2.weight", "0.5.3.bn2.bias", "0.5.3.bn2.running_mean", "0.5.3.bn2.running_var", "0.6.0.conv1.weight", "0.6.0.bn1.weight", "0.6.0.bn1.bias", "0.6.0.bn1.running_mean", "0.6.0.bn1.running_var", "0.6.0.conv2.weight", "0.6.0.bn2.weight", "0.6.0.bn2.bias", "0.6.0.bn2.running_mean", "0.6.0.bn2.running_var", "0.6.0.downsample.0.weight", "0.6.0.downsample.1.weight", "0.6.0.downsample.1.bias", "0.6.0.downsample.1.running_mean", "0.6.0.downsample.1.running_var", "0.6.1.conv1.weight", "0.6.1.bn1.weight", "0.6.1.bn1.bias", "0.6.1.bn1.running_mean", "0.6.1.bn1.running_var", "0.6.1.conv2.weight", "0.6.1.bn2.weight", "0.6.1.bn2.bias", "0.6.1.bn2.running_mean", "0.6.1.bn2.running_var", "0.6.2.conv1.weight", "0.6.2.bn1.weight", "0.6.2.bn1.bias", "0.6.2.bn1.running_mean", "0.6.2.bn1.running_var", "0.6.2.conv2.weight", "0.6.2.bn2.weight", "0.6.2.bn2.bias", "0.6.2.bn2.running_mean", "0.6.2.bn2.running_var", "0.6.3.conv1.weight", "0.6.3.bn1.weight", "0.6.3.bn1.bias", "0.6.3.bn1.running_mean", "0.6.3.bn1.running_var", "0.6.3.conv2.weight", "0.6.3.bn2.weight", "0.6.3.bn2.bias", "0.6.3.bn2.running_mean", "0.6.3.bn2.running_var", "0.6.4.conv1.weight", "0.6.4.bn1.weight", "0.6.4.bn1.bias", "0.6.4.bn1.running_mean", "0.6.4.bn1.running_var", "0.6.4.conv2.weight", "0.6.4.bn2.weight", "0.6.4.bn2.bias", "0.6.4.bn2.running_mean", "0.6.4.bn2.running_var", "0.6.5.conv1.weight", "0.6.5.bn1.weight", "0.6.5.bn1.bias", "0.6.5.bn1.running_mean", "0.6.5.bn1.running_var", "0.6.5.conv2.weight", "0.6.5.bn2.weight", "0.6.5.bn2.bias", "0.6.5.bn2.running_mean", "0.6.5.bn2.running_var", "0.7.0.conv1.weight", "0.7.0.bn1.weight", "0.7.0.bn1.bias", "0.7.0.bn1.running_mean", "0.7.0.bn1.running_var", "0.7.0.conv2.weight", "0.7.0.bn2.weight", "0.7.0.bn2.bias", "0.7.0.bn2.running_mean", "0.7.0.bn2.running_var", "0.7.0.downsample.0.weight", "0.7.0.downsample.1.weight", "0.7.0.downsample.1.bias", "0.7.0.downsample.1.running_mean", "0.7.0.downsample.1.running_var", "0.7.1.conv1.weight", "0.7.1.bn1.weight", "0.7.1.bn1.bias", "0.7.1.bn1.running_mean", "0.7.1.bn1.running_var", "0.7.1.conv2.weight", "0.7.1.bn2.weight", "0.7.1.bn2.bias", "0.7.1.bn2.running_mean", "0.7.1.bn2.running_var", "0.7.2.conv1.weight", "0.7.2.bn1.weight", "0.7.2.bn1.bias", "0.7.2.bn1.running_mean", "0.7.2.bn1.running_var", "0.7.2.conv2.weight", "0.7.2.bn2.weight", "0.7.2.bn2.bias", "0.7.2.bn2.running_mean", "0.7.2.bn2.running_var", "1.2.weight", "1.2.bias", "1.2.running_mean", "1.2.running_var", "1.4.weight", "1.4.bias", "1.6.weight", "1.6.bias", "1.6.running_mean", "1.6.running_var", "1.8.weight", "1.8.bias".
        Unexpected key(s) in state_dict: "opt_func", "loss_func", "metrics", "true_wd", "bn_wd", "wd", "train_bn", "model_dir", "callback_fns", "cb_state", "model", "data", "cls".

My fastai version is same as fastai version in Google colab but I still get the same problem.I expect my model to be deployed on my local server.

I copied this from the question that I posted at StackOverflow a few days ago but seems no one help me. Please help me figure out what wrong with this. Thank you so much!

mrfabulous1 · October 7, 2019, 6:17am

Good morning ApacheT101 hope you are well!

In my experience of creating at least 20 apps between Google Colab and Anaconda on my desktop, it is not possible for the app to work unless your Anaconda environment matches the Colab environment 100%.

In practice this means that when you do !pip list on Colab where you trained your model. All the files library versions used in the requirements.txt file in the app, must be changed to match versions you received when you did !pip list on Colab.
If you try and run a model/app on your desktop in an Anaconda environment the app will fail, unless all the file versions numbers in your anaconda environment match the Colab versions.
Sometimes point 2 even fails as libraries can be security patched or have minor changes that haven’t been announced yet and don’t have a version number associated with them.
So it means you need to create a new anaconda environment each time you train a model on Google Colab and any library versions are different to your current Anaconda environment.
If you change your current Anaconda environment library versions, then all your existing apps created with that environment may/will break.

In summary create new environment every time library/libraries on Colab differ from libraries on Anaconda.

Cheers mrfabulous1
ps. I now use Docker to overcome this issue.

Alican1 · October 7, 2019, 9:29am

This worked for me as well thanks! I use paperspace.

NickBenz · October 8, 2019, 6:31am

Hey guys! Please help

Trying to run:

If you have Docker installed, you can test your app in the same environment as Render’s by running the following command at the root of your repo:

docker build -t fastai-v3 . && docker run --rm -it -p 5000:5000 fastai-v3

docker builds but app doesn’t work. After upload image and push Analyze I got

http://0.0.0.0:5000/analyze
POST
500 Internal Server Error

and 'Conv2d' object has no attribute 'padding_mode' in terminal

Does anybody know how to fix it? I’m already smoked half of Google but can’t find answer

mrfabulous1 · October 8, 2019, 10:01am

Hi NikitaBeznosikov hope your having a jolly day!

Which platform did you train your model on?
Which platform are you running your app on?
Does your app work before putting it in to Docker?

Cheers mrfabuloous1

ApacheT101 · October 8, 2019, 1:00pm

Hi mrfabulous1, I am so grateful that you replied to my post, but sadly, today I used so much time at deploy model but in the end, many problems occurred, here are the problems:

Docker: I tried to use Docker but this occurred.

and I don’t think I can use Docker since I need to pay to install Windows 10 Pro and do you have any guide on deploying model by using Docker?

Google Cloud Platform: I am able to use GCP since I have a 1-year free trial membership but the same error( Missing key(s) in state_dict: “0.0.weight”, “0.1.weight”, “0.1.bias”…) popped up again at Google Cloud Shell, do you have any solution for this if I have no choice but to use this?

Thanks a lot!

NickBenz · October 8, 2019, 4:22pm

Which platform did you train your model on?

I used Paperspace + Fast.AI 1.0 (V3)

Which platform are you running your app on?

Trying to run it locally

Does your app work before putting it in to Docker?

I’ve never tried to run it somewhere else but locally

Also should I have an Nvidia card on my machine to run locally?

wgpubs · October 8, 2019, 4:56pm

Will there be the ability to spin up a GPU instance on Render?

I really love the cron jobs and want to use it to pull several thousand text examples at a time for inference against a couple NLP models that really need a GPU.

Thanks (and congrats on Disrupt!!!)

NickBenz · October 9, 2019, 7:07am

I fixed it. Changed requirements.txt to this

aiofiles==0.4.0
aiohttp==3.5.4
asyncio==3.4.3
fastai==1.0.52
torch==1.0.0
torchvision==0.4.0
numpy==1.16.3
starlette==0.12.0
uvicorn==0.7.1
python-multipart==0.0.5

problem was here

before:

torch==1.1.0
torchvision==0.3.0

after:

torch==1.0.0
torchvision==0.4.0