Model Deployment GCP: cannot download model, too big?

Part 1 (2019)
1

Hello,

I followed the instructions for GCP model deployment but I am getting UnpicklingError: invalid load key, ‘<’ error which according to this thread is connected to inability to download my saved model.

My model is 200+ M and I uploaded it to google drive. I get the link, but cannot get direct download and bypass this “large file cannot scan for viruses” warning. I tried to use GCP shell and wget to download model file directly to models folder, but get the error again, most likely due to this Google drive large file warning.

I also tried to modify the link via https://rawdownload.now.sh/ and still arrive at that page with large file warning and direct download does not work.

Did anyone have the similar issues? Is there any workaround this?

2

Ok, so I think I uploaded the model via onedrive, but I still get the following error:

Step 7/9 : RUN python app/server.py ---> Running in 110a3eebbb9f Traceback (most recent call last): File "app/server.py", line 39, in <module> learn = loop.run_until_complete(asyncio.gather(*tasks))[0] File "/usr/local/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete return future.result() File "app/server.py", line 34, in setup_learner learn.load(model_file_name) File "/usr/local/lib/python3.6/site-packages/fastai/basic_train.py", line 267, in load state = torch.load(source, map_location=device) File "/usr/local/lib/python3.6/site-packages/torch/serialization.py", line 426, in load return _load(f, map_location, pickle_module, **pickle_load_args) File "/usr/local/lib/python3.6/site-packages/torch/serialization.py", line 603, in _load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '\x0d'. The command '/bin/sh -c python app/server.py' returned a non-zero code: 1 ERROR ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: exit status 1

My server.py file is here what am I doing wrong?

3

Hey! Has anyone resolved this issue?

Traceback (most recent call last):
File “app/server.py”, line 37, in
learn = loop.run_until_complete(asyncio.gather(*tasks))[0]
File “/usr/local/lib/python3.6/asyncio/base_events.py”, line 488, in run_until_complete
return future.result()
File “app/server.py”, line 32, in setup_learner
learn.load(model_file_name)
File “/usr/local/lib/python3.6/site-packages/fastai/basic_train.py”, line 269, in load
state = torch.load(source, map_location=device)
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 529, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File “/usr/local/lib/python3.6/site-packages/torch/serialization.py”, line 692, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, ‘<’.
The command ‘/bin/sh -c python app/server.py’ returned a non-zero code: 1
ERROR
ERROR: build step 0 “gcr.io/cloud-builders/docker” failed: exit status 1

I searched around but couldn’t find any solutions… Any help appreciated!

4

In my case it was the problem with downloading the model, so I uploaded it to the dropbox (had to open business account) and it worked. The issue was that download had to start automatically, but in all previous download versions google kept asking for confirmation to download or was telling about this large file viruses check thing. Check your model link, if it downloads automatically, this link will work, in my case dropbox link did the trick. Your error looks similar, so maybe it will help you.

2 Likes
5

Thanks! that worked. I was using Google drive and it had an additional prompt which was preventing the download. Thanks again.

1 Like