Problem installing fastai on Jupyter notebook running on Google Cloud

fastai
1

Hello, I am new to fastai course, and after few hours scanning through the documentation and the forum I am looking for help.
I am following up the instructions of https://course19.fast.ai/start_gcp.html and cannot properly install the fastai library after running the command on the terminal:

conda install -c fastai fastai

i receive the error message:

ERROR conda.core.link:_execute(700): An error occurred while installing package ‘defaults::pytorch-1.3.1-cpu_py37h62f834f_0’.

Rolling back transaction: done

[Errno 13] Permission denied: ‘/opt/conda/lib/python3.7/site-packages/torch/pycache/config.cpython-37.pyc’

I thought this was only warning messages but when running the notebooks for the first lesson, also got error message related to the fastai library function that could not be found…I would appreciate any help, and sorry if this issue is trivial.

Thanks

2

Any luck with this yet? I’m having the same issue.

3

No luck yet. My turn around in the meantime has been to used google colab instead of setting up google cloud server, all libraries are already installed, no setup needed…

But still I hope to have a response to setting google cloud instance and installed the library properly,

1 Like
4

Just got it to work. Here’s what worked for me:
After launching the terminal, I uninstalled torch and torchvision:
pip uninstall torch
pip uninstall torchvision

Then install fastai:
conda install -c fastai -c pytorch -c anaconda fastai gh anaconda

5

I tried the same, removed successfully torch and torch vision, however, could not install fastai successfully and got the error message below. I can’t understand what to do next, how to remove the specified file manually…would be great to have help on this.

"Executing transaction: - WARNING conda.gateways.disk.delete:unlink_or_rename_to_trash(140): Could not remove or rename /opt/conda/lib/python3.7/site-packages/torch/pycache/init.cpython-37.pyc. Please remove this file manually (you may need to reboot to free file handles)
done
ERROR conda.core.link:_execute(700): An error occurred while installing package ‘pytorch::pytorch-1.0.0-py3.7_cuda9.0.176_cudnn7.4.1_1’.
Rolling back transaction: done

[Errno 13] Permission denied: ‘/opt/conda/lib/python3.7/site-packages/torch/pycache/init.cpython-37.pyc’
()

6

try first doing sudo rm /opt/conda/lib/python3.7/site-packages/torch/pycache/init.cpython-37.pyc

7

I tried this and got an error message that this file “/opt/conda/lib/python3.7/site-packages/torch/pycache/init.cpython-37.pyc” does not exist. this is strange since the previous error message mentioned that this file needed to be removed manually before continuing with the installation of the fastai library.

Thanks for the help and sorry for the trouble, hope to resolve this problem,

8

For me, it worked when I used the default python3 image with Cuda 11 rather than the pytorch 1.6 version. It let me install fastai normally (also I think the instructions are a bit outdated, you can install fastbook using conda now I think).

I think maybe the problem with the 1.6 image is that the new version of fastai tries to install pytorch 1.7, and the image must have some restriction on the pytorch version installed (maybe since it’s explicitly pytorch 1.6)

9

Thanks Allen,

How do you use “the default python3 image with Cuda 11 rather than the pytorch 1.6 version”? How is it related to the below command that does not install for me?

conda install -c fastai fastai

Thanks!

10

When you are creating the instance, there are several options, some of them with tensor flow or pytorch preinstalled. Don’t choose the pytorch image, just go with the base image with cuda 11 and you should be fine. It must be that the pytorch image has special permissions so you can’t modify the pytorch version (1.6) where the latest fastai uses 1.7

11

Thanks Allen,

What is the base instance image with cuda 11 you refer to? I went through the list of image in GCP and would this one be ok? [c0-deeplearning-common-cu110-v20201105-debian-10]. would that be fine?

12

I think so, you can just try it out and see if it works. If it doesn’t it’s pretty easy to delete afterwards.

13

After spending too much time, I could not find how to change the image for the instance…I used the CLI commands below (as in the GCP fastai guide) to install the instance with IMAGE_FAMILY=“pytorch-latest-gpu”. What other IMAGE_FAMILY should I use? Thanks for the help!

export IMAGE_FAMILY="pytorch-latest-gpu" 
export INSTANCE_NAME="my-fastai-instance"
export INSTANCE_TYPE="n1-highmem-8" # budget: "n1-highmem-4"

# budget: 'type=nvidia-tesla-t4,count=1'
gcloud compute instances create $INSTANCE_NAME \
        --zone=$ZONE \
        --image-family=$IMAGE_FAMILY \
        --image-project=deeplearning-platform-release \
        --maintenance-policy=TERMINATE \
        --accelerator="type=nvidia-tesla-p100,count=1" \
        --machine-type=$INSTANCE_TYPE \
        --boot-disk-size=200GB \
        --metadata="install-nvidia-driver=True" \
        --preemptible
14

ok, I finally got it to work! in case it can help anyone with the same problem, I changed the IMAGE_FAMILY to “pytorch-1-4-cu101-debian-10” as follows:

export IMAGE_FAMILY=“pytorch-1-4-cu101-debian-10”
export ZONE=“us-central1-f”
export INSTANCE_NAME=“my-fastai-instance”
export INSTANCE_TYPE=“n1-highmem-8”

gcloud compute instances create $INSTANCE_NAME
–zone=$ZONE
–image-family=$IMAGE_FAMILY
–image-project=deeplearning-platform-release
–maintenance-policy=TERMINATE
–accelerator=“type=nvidia-tesla-p100,count=1”
–machine-type=$INSTANCE_TYPE
–boot-disk-size=200GB
–metadata=“install-nvidia-driver=True”
–preemptible

this finally worked after many trial and errors, not sure why the other IMAGE did not work! Quite frustrating at the end. I hope this helps someone.

15

Solution that worked for me:
To the problem of permission denied to any file in ‘/opt/conda/lib/python3.7/site-packages/torch/…’

Try changing the folder permissions to read-write for everyone recursively (to all sub folders in folder)
sudo chmod -R a+rwx /opt/conda/lib/python3.7/site-packages/torch/

2 Likes
16

This worked for me…thanks!