Deployment Platform: Render ✅

Hello everyone, I am trying to deploy my first classifier on Render.

I decided to install the bear classifier example in the production section of the course first, to reduce any issues I may have when installing my app.
Unfortunately when I press the deploy button on the Render Services page I receive the following error.

Jun 8 06:33:31 AM INFO[0195] RUN python app/server.py
Jun 8 06:33:34 AM Traceback (most recent call last):
File “app/server.py”, line 9, in
from fastai.vision import *
File “/usr/local/lib/python3.7/site-packages/fastai/vision/init.py”, line 3, in
from .learner import *
File “/usr/local/lib/python3.7/site-packages/fastai/vision/learner.py”, line 6, in
from . import models
File “/usr/local/lib/python3.7/site-packages/fastai/vision/models/init.py”, line 2, in
from torchvision.models import ResNet,resnet18,resnet34,resnet50,resnet101,resnet152
File “/usr/local/lib/python3.7/site-packages/torchvision/init.py”, line 1, in
from torchvision import models
File “/usr/local/lib/python3.7/site-packages/torchvision/models/init.py”, line 11, in
from . import detection
File “/usr/local/lib/python3.7/site-packages/torchvision/models/detection/init.py”, line 1, in
from .faster_rcnn import *
File “/usr/local/lib/python3.7/site-packages/torchvision/models/detection/faster_rcnn.py”, line 7, in
from torchvision.ops import misc as misc_nn_ops
File “/usr/local/lib/python3.7/site-packages/torchvision/ops/init.py”, line 1, in
from .boxes import nms, box_iou
File “/usr/local/lib/python3.7/site-packages/torchvision/ops/boxes.py”, line 2, in
from torchvision import _C
ImportError: libcudart.so.9.0: cannot open shared object file: No such file or directory
Jun 8 06:33:35 AM error building image: error building stage: waiting for process to exit: exit status 1
Jun 8 06:33:35 AM error: exit status 1

Does anyone have the solution to this problem or can someone suggest the best approach to solving this issue?
Thanks in advance.smiley::smiley:

I was using render too, now i switched over to Google App Engine. There is a tutorial on the course website on it, and not a bad way to make use of the free compute credits Google is providing for free.

3 Likes

Hi,

I am trying to deploy my model on Render, but getting the following error:
File “app/server.py”, line 48, in
learn = loop.run_until_complete(asyncio.gather(*tasks))[0]
File “/usr/local/lib/python3.7/asyncio/base_events.py”, line 584, in run_until_complete
return future.result()
File “app/server.py”, line 35, in setup_learner
learn = load_learner(path, export_file_name)
File “/usr/local/lib/python3.7/site-packages/fastai/basic_train.py”, line 598, in load_learner
state = torch.load(source, map_location=‘cpu’) if defaults.device == torch.device(‘cpu’) else torch.load(source)
File “/usr/local/lib/python3.7/site-packages/torch/serialization.py”, line 387, in load
return _load(f, map_location, pickle_module, **pickle_load_args)
File “/usr/local/lib/python3.7/site-packages/torch/serialization.py”, line 564, in _load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, ‘<’.

Can anybody help?

I found somewhere that the following line of code should be used for pickle error :

from functools import partial
import pickle
pickle.load = partial(pickle.load, encoding=“latin1”)
pickle.Unpickler = partial(pickle.Unpickler, encoding=“latin1”)
model = torch.load(model_file, map_location=lambda storage, loc: storage, pickle_module=pickle

However now I m getting the following error now:
Jun 11 03:41:50 PM File “app/server.py”, line 17
Jun 11 03:41:50 PM export_file_url = ‘https://drive.google.com/uc?export=download&id=1-pQHp5iT5BiuyBchBq0yBYQIzmsM9is9
Jun 11 03:41:50 PM ^
Jun 11 03:41:50 PM error building image: error building stage: waiting for process to exit: exit status 1
Jun 11 03:41:50 PM SyntaxError: invalid syntax
Jun 11 03:41:50 PM error: exit status 1

What am I doing wrong? I have followed exactly the same instructions provided?

I had the same error.
Reuploading the model weights to drive/dropbox solved it.

I uploaded my export.pkl file to Google Drive and updated all the requisite libraries to match with the render requirements.txt and was still receiving the error:

_pickle.UnpicklingError: invalid load key, '<'.

After running the docker image locally I finally found the problem was because my .pkl file was over 25Mb. Google Drive prompts large download attempts with a warning page about virus scanning (even using that confirmation link as the download url fails!). The html warning page is downloaded instead

<!DOCTYPE html><html><head><title>Google Drive - Virus scan warning</title>
...
</html>

hence the invalid key <!
My solution was to upload to Dropbox where I didn’t have any problems…

If you must use Google Drive, use the Drive API:
https://developers.google.com/drive/api/v3/manage-downloads

3 Likes

Hello Adam,

I have the same error, but as it happens I did try dropbox as well but still got the same issue (see below).

How did you manage to get the full error description? I just want to confirm it’s a size error. I am using a free dropbox account, but I don’t think there is any limitations there. The link I have take me to a page where you have to click download. I’m wondering if that might the issue?

Regards

Dave

Ok, figured out what I was doing wrong!
The dropbox URL link was set with 0 at the end, and not 1. Set to one it allowed the file to download straight away

https://www.dropbox.com/s/jjvxmxe95pemfhc/cars99?dl=1

Dave

2 Likes

Getting the following error when deploying to render… any ideas?

  File "app/server.py", line 48, in <module>
    learn = loop.run_until_complete(asyncio.gather(*tasks))[0]
  File "/usr/local/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "app/server.py", line 35, in setup_learner
    learn = load_learner(path, export_file_name)
  File "/usr/local/lib/python3.7/site-packages/fastai/basic_train.py", line 600, in load_learner
    src = LabelLists.load_state(path, state.pop('data'))
KeyError: 'data'

My model uses a PTH file and I see the export file name is a .pkl could that be the issue??

Did you ever get a solve for this? I am running in to a similar issue

am also having this error. can anyone help with this please.
have configure the requirement.txt and yet thesame error

please i need help. having being trying to deploy to render for some days now. and have tried every suggestion being made here, but still not working.

please what did you think might be happening.

Hi Stephen,

Where did you get that requirements.txt from? Did you create it yourself? It has different versions of things compared with the example from Anurag and co. here.
Having Googled your error messages (about numpy _multiarray_umath), I found this here saying that _multiarray_umath is new in numpy 1.16.0, whereas your requirements.txt asked for 1.15.1. That might be the problem?

I suggest trying to follow/use the example repo’s requirements.txt first, see if it works.

Good luck!

Yijin

Thanks.
I got the requirements.txt from pip list and some are already provided I’m d repo
I only change the torch version and numpy

I will change the numpy version to 1.16.0 and see if any change and also try the example.

Thanks

Hey - I was able to sort out this issue if you are still seeing it, there is an issue with the model file of some sort. I was passing off a PTH file instead of a PKL file to the server which caused the issue. Might be the same but the issue is that the pop function can’t find the key data

1 Like

have update the numpy and it has stopped showing the error. but it keeps showing the pickle error

and then i googled how to solve it, but the solution involved the encoding type to pickle
sol

is there a way to solve it. Thanks

has this been solved?

have found a way to go about the application, mine is now working. how i go about it:

  1. i fork the fastai-v3-render app repo for teddy bear again, connect it to the render web service and deploy
    to check if is working. it actually work.
  2. since the teddy bear app is working, i followed the direction from the top post by @aschiller here . for file whose mb is > 25 mb might be seen as virus, so i generate a google cloud api with the shared link of the export.pkl. follow the step here.
  3. then i copied the generated link to the teddy bear server.py to replace the former export.pkl and change the class name. and deploy on render, it load properly, and it gave error that the nn.Module is different
  4. so i changed the fastai verion to my own version 1.0.54 and torch to 1.0.0 and torch vision to torchvision=0.2.1.
    5)then it was deployed again and it work perfectly
2 Likes

Hooray :laughing::laughing::laughing::laughing:I have now got my model working on Render.

I train my models on Google Colab as I do not have a GPU at the present and my mac takes to long sometimes.

When I forked the GitHub teddy bear example I mentioned in my previous post I had an error.

My Solution
l looked in the requirements.txt and changed the pytorch version to the version in my Google Colab notebook.

I got a different error.

I ran !pip list on my Google Colab notebook and changed all the libraries in the requirements.txt to the same version as on my Colab notebook.

This solved the problem.

However even though the errors were gone in the Render terminal the app was stuck in the analyzing state.

I switched from Safari to Chrome and it works fine now.

It seems that the .pkl must match the libraries of the system the .pkl was created on, so in this case the requirements.txt must be amended.

Hope this helps.

3 Likes

hi, I have this error during deployment. I can not figure it out.

i already edit the requirements.txt to match my environment.
aiofiles==0.4.0
aiohttp==3.5.4
asyncio==3.4.3
fastai==1.0.53
https://download.pytorch.org/whl/cpu/torch-1.1.0-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/cpu/torchvision-0.3.0-cp37-cp37m-linux_x86_64.whl
numpy==1.16.4
starlette==0.12.1
uvicorn==0.8.2
python-multipart==0.0.5

I attached part of my model in case , it is required.

thanks a lot.