ended up doing this:
I couldn’t really find a way to really use fast.ai’s databunch with google cloud storage
I am trying to deploy the bear model on google cloud platform. but i am keep getting this error complaining “can not allocate memory”. Have anybody run into the same problem and have some insight on this:
Step #1 - “builder”: INFO tar_runtime_package took 23 seconds
Step #1 - “builder”: INFO starting: gzip_tar_runtime_package
Step #1 - “builder”: INFO gzip_tar_runtime_package gzip /tmp/tmpuJcM44.tar -1
Step #1 - “builder”: INFO gzip_tar_runtime_package took 0 seconds
Step #1 - “builder”: INFO building_python_pkg_layer took 29 seconds
Step #1 - “builder”: INFO uploading_all_package_layers took 34 seconds
Step #1 - “builder”: INFO build process for FTL image took 258 seconds
Step #1 - “builder”: INFO full build took 258 seconds
Step #1 - “builder”: ERROR gzip_tar_runtime_package gzip /tmp/tmpuJcM44.tar -1
Step #1 - “builder”: exited with error [Errno 12] Cannot allocate memory
Step #1 - “builder”: gzip_tar_runtime_package is likely not on the path
Step #1 - “builder”: Traceback (most recent call last):
Step #1 - “builder”: File “/usr/lib/python2.7/runpy.py”, line 174, in _run_module_as_main
Step #1 - “builder”: “main”, fname, loader, pkg_name)
Step #1 - “builder”: File “/usr/lib/python2.7/runpy.py”, line 72, in _run_code
Step #1 - “builder”: exec code in run_globals
Step #1 - “builder”: File “/usr/local/bin/ftl.par/main.py”, line 65, in <module>
Step #1 - “builder”: File “/usr/local/bin/ftl.par/main.py”, line 60, in main
Step #1 - “builder”: File “/usr/local/bin/ftl.par/main/ftl/common/ftl_error.py”, line 77, in InternalErrorHandler
Step #1 - “builder”: IOError: [Errno 2] No such file or directory: ‘""/output’ Finished
Step #1 - “builder” ERROR ERROR: build step 1 “gcr.io/gae-runtimes/python37_app_builder:python37_20190527_3_7_3_RC00” failed: exit status 1
Hello,
Does anybody know what is the best zone to choose for google cloud platform.
I’m from the Netherlands and picked the Amsterdam zone, that sounded the most logical to me.
I got everything working and started the first class, but I can’t start the VM half of the times because there are no resources available.
Are there differences in availability between zones and is is maybe handier to use the US-west zone as stated in the set up guide?
I used europe-west4-b, that worked for me.
However, a lot of the times there are not enough resources available and I can’t start my instance.
1 Like
use the normal instance, not the pre-emptied. It’s much more expensive, but you have the $300 gcp credit. You connect with only one try and doesn’t disconnect
Hi, I really met a problem, and I don’t know to process…
I used the “gcloud compute ssh –zone=us-west1-b jupyter@my-fastai-instance – -L 8080:localhost:8080”, but it said no jupyter can be recognized, can you help me?
Are you using an instance you created following the steps here? Can you give me a bit more info on your setup? Thanks.
Yes, I followed the steps as the https://course.fast.ai/start_gcp.html. All goes well until the last step. I can’t use the juypter notebook. It cannot be connected. I don’t know why. So I tried many other ways, like https://github.com/arunoda/fastai-shell. It’s very easy, but I met the same problem, just as shown in the picture. Would you mind to communicate with email? My email is caiwangzheng@gmail, I have tried to use google cloud platform to perform fastai for two weeks… I don’t know how to solve it… Really thank you for your reply…
1 Like
I’m sorry, this area isn’t my strength so I may not be the best person to help. Looking very closely at the commands in the first screenshot, it looks like you used only one dash -zone=us-west1-b instead of two dashes --zone=us-west-1b. When I tried it I got a similar error message to yours about unrecognized arguments. Hopefully that helps.
@MadeUpMasters @Timothy_ZHENG, it looks like I’m running into a similar issue to you. I’m looking to launch a jupyter notebook that is connected to a Google Cloud instance. I have followed all the steps in https://course.fast.ai/start_gcp.html, but I keep on getting an error when I issue the gcloud compute ssh command. Any idea what I’m missing here? Thanks!
Error Message:
ssh: connect to host 35.233.250.67 port 22: Resource temporarily unavailable
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].
export IMAGE_FAMILY="pytorch-latest-gpu"
export ZONE="us-west1-b"
export INSTANCE_NAME="fastai-instance"
export INSTANCE_TYPE="n1-highmem-4"
gcloud compute instances create $INSTANCE_NAME \
--zone=$ZONE \
--image-family=$IMAGE_FAMILY \
--image-project=deeplearning-platform-release \
--maintenance-policy=TERMINATE \
--accelerator="type=nvidia-tesla-k80,count=1" \
--machine-type=$INSTANCE_TYPE \
--boot-disk-size=200GB \
--metadata="install-nvidia-driver=True" \
--preemptible
export ZONE="us-west1-b"
export INSTANCE_NAME="fastai-instance"
gcloud compute ssh --zone=$ZONE jupyter@$INSTANCE_NAME -- -L 8080:localhost:8080
I think this means there are no premptible instances available. You can try again later, that might work
I am sorry but I haven’t solved this problem. I am struggling with it. We can communicate if I solve it
Continuing the discussion from Google Cloud Platform:
[quote=“floristco, post:38, topic:35907, full:true”]
I think this means there are no premptible instances available. You can try again later, that might work
[/quote]
It all seems good, but I can’t open the Jupyter Notebook
If you connect with gccloud jupyter notebook is already running and routed to your localhost. You should see your notebook here: localhost:8080/
I was trying to boot my gcloud preemptible VM, but before the VM properly boots up and I start my Jupyter Notebook, the VM gets auto shut down due to the preemptibe option. Did anyone else feel the same? My VM is in the US West region.
I am seeing an issue with preemptible instances as well. As soon as I add the --preemptible flag, the instance does not start
ERROR: (gcloud.compute.instances.create) Could not fetch resource:
- Instance failed to start due to preemption.
If I remove the flag, the instance starts ok. Its more expensive though. Why does the preemptible instance not start up? Any help is appreciated. Thanks
Preemptible instance means it will use whatever resources are available in that area but priority is placed on non-preemptible resources. It’s effectively what other extra resources that are available are given to you. But sometimes there are no extra idle resources:
A preemptible VM is an instance that you can create and run at a much lower price than normal instances. However, Compute Engine might terminate (preempt) these instances if it requires access to those resources for other tasks. Preemptible instances are excess Compute Engine capacity, so their availability varies with usage.
(From GCP help page)
1 Like
I have same problem.
I think it is because google doesn’t allow us to do so with free tier. See below:
Preemptible instance limitations
Preemptible instances function like normal instances but have the following limitations:
- The Google Cloud Free Tier credits for Compute Engine do not apply to preemptible instances.