Setup help

It seems you installed it in pytorch_p36 conda environment? or you have created a new env like described in the repo? (https://github.com/fastai/fastai2)

One thing to note, when you restart SM notebook - the preinstalled environments, including packages and conda will revert back to original ones (as if you just created a notebook).

It happenes because user volume is mounted to $HOME/SageMaker folder, everything above it will be reinitialized if you stop and start notebook instance.

The way to make it persistent is to use lifecycle configurations: https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html#nbi-lifecycle-config-install

Also,
Cloud Formation template from this tutorial https://course.fast.ai/start_sagemaker.html can be updated by just replacing old fastai installation steps, with new fastai installation steps.
For that you need to go to https://course.fast.ai/start_sagemaker.html, download template locally first and then edit yaml file (i mean this one https://s3-eu-west-1.amazonaws.com/mmcclean-public-files/sagemaker-fastai-notebook/sagemaker-cfn.yml)

Thanks @tensoralex I did edit the yml file from there – but only replaced the pip install fastai to fastai2.
Did not change anything else. I will follow the instructions in https://github.com/fastai/fastai2 for the new env and try to run it. Thanks a bunch!

1 Like

Thanks for all your help. I was able to create a notebook on AWS Sagemaker using Cloudfront script.
Here is the script that I used to create a notebook instance in us-east-1

Plz make sure you requested access to ml.p2.xlarge before using this as documented here:https://course.fast.ai/start_sagemaker.html

2 Likes

FYI:

I had an issue here; I conda installed graphviz which installed version 2.40.2. which jupiter failed to import. I then turned to pip install graphviz which installed version 0.13.2. This worked.

Error running notebook 02_production: PermissionDenied

returned from image search.

At code cell 2 key is set to ‘XXX’ guess that’s an issue??

I guess we already discussed that one here: https://forums.fast.ai/t/02-production-permissiondenied-error/65823

(Adding it for others to find, if needed.)

Francesco,

I didn’t try SageMaker, but I will probably pursue it today. I ended up doing Colab Pro, but I’m pretty dissatisfied with it since I can’t easily flip working code to a web app the way i would like to.

The EC2 images issue I had seemed to be related to a CUDA mismatch, and this occurred upon using the conda create command off of the environment.yaml file in the fastai2 repo. Conda installs Pytorch 1.4 (due to torchvision 0.5 requirement) and it seems like it went ahead and updated the cuda packages and blew up my compatibility. No idea for sure though.

Have you used SageMaker before? I haven’t tried it, but I guess I will give it a whirl this evening and will let you know if I get it rolling.

Just looking at Google Cloud options last night, and they are radically cheaper than AWS, which further sours me on Amazon for this class.

Edit: I will give the instructions referenced by @pinaki a try.

I agree with you about the feeling towards AWS. Without doubt it is quite expensive and not super user-friendly.
I have used SageMaker extensively and that is why I wanted to hear about potential incompatibilities with fastai2, before diving into a potential bloodshed :). I am particularly interested in testing SageMaker Studio, and I was hoping to get it to work.
As you suggest, @pinaki’s instructions seem quite promising, though, and I will try them out too!
Thanks and have a good one!

Note that my recommendation in the top post of this thread is to use Paperspace Gradient. There’s a free option, an inexpensive monthly subscription option, and also paid per-minute options. Unlike Colab, it’s “real” Jupyter, and everything works in it nicely. There’s also folks from Paperspace here helping out.

8 Likes

If anyone is interested in trying out running course notebooks on Kaggle platform, I have created a simple starter notebook, which you can copy and edit:
https://www.kaggle.com/gbecon/fastai2-starter

Note that Kaggle provides 30 hours of GPU time per week for free, but you have to enable GPU and internet connection to be able to run the notebook.

6 Likes

Thanks @jeremy

I’m going to go ahead and use that instead of Colab.

Be sure to let us know how that goes for you! :slight_smile:

@jeremy

I work for an up and coming enterprise analytics company (Alteryx), so I place a huge value on UX that walks the line between simplicity and customizability. I was pretty blown away by how low the cognitive load was to get the system running. Love the built in terminal! Now I’m getting excited about their Core capabilities for my day job. :slight_smile:

1 Like

Works great thank you !

I have a question about paperspace.

Is there tight integration between the notebooks & /storage via Gradient and their dedicated machines via Core? I’m wondering if I can spin up a notebook when I want with gradient or choose to spin up a whole machine to do some things in the terminal and still have access to anything I saved in /storage while experimenting with a notebook? Also would fastaiv2 be available on their dedicated machines?

My thought is I’d like a dedicated machine to spin up and develop stuff on for a personal project when needed but still have access to my experimental notebooks that I end up running here and there throughout the course.

Thanks!

1 Like

I just published a setup guide for AWS here

3 Likes

I just published a setup guide for AWS here. It uses a SageMaker Notebook Instance. Will hopefully get a SageMaker Studio version working soon…

2 Likes

Happy to help out! Just added a SageMaker setup guide :slight_smile:

2 Likes

This is great, thanks!
I managed to get a Sagemaker notebook instance to work with fastai2 following @pinaki’s instructions.
Having Studio too would be awesome.
I investigated it quickly yesterday but I was put away by this page.
Specifically this part


Maybe I am missing something super basic in the UI, but I could not find a way to set the instance type.
It apparently defaults to ml.t3.medium.
Super weird…

Awesome Matt - I’m so happy to see you here! :slight_smile: