Setup help

I agree with you about the feeling towards AWS. Without doubt it is quite expensive and not super user-friendly.
I have used SageMaker extensively and that is why I wanted to hear about potential incompatibilities with fastai2, before diving into a potential bloodshed :). I am particularly interested in testing SageMaker Studio, and I was hoping to get it to work.
As you suggest, @pinaki’s instructions seem quite promising, though, and I will try them out too!
Thanks and have a good one!

Note that my recommendation in the top post of this thread is to use Paperspace Gradient. There’s a free option, an inexpensive monthly subscription option, and also paid per-minute options. Unlike Colab, it’s “real” Jupyter, and everything works in it nicely. There’s also folks from Paperspace here helping out.

8 Likes

If anyone is interested in trying out running course notebooks on Kaggle platform, I have created a simple starter notebook, which you can copy and edit:
https://www.kaggle.com/gbecon/fastai2-starter

Note that Kaggle provides 30 hours of GPU time per week for free, but you have to enable GPU and internet connection to be able to run the notebook.

6 Likes

Thanks @jeremy

I’m going to go ahead and use that instead of Colab.

Be sure to let us know how that goes for you! :slight_smile:

@jeremy

I work for an up and coming enterprise analytics company (Alteryx), so I place a huge value on UX that walks the line between simplicity and customizability. I was pretty blown away by how low the cognitive load was to get the system running. Love the built in terminal! Now I’m getting excited about their Core capabilities for my day job. :slight_smile:

1 Like

Works great thank you !

I have a question about paperspace.

Is there tight integration between the notebooks & /storage via Gradient and their dedicated machines via Core? I’m wondering if I can spin up a notebook when I want with gradient or choose to spin up a whole machine to do some things in the terminal and still have access to anything I saved in /storage while experimenting with a notebook? Also would fastaiv2 be available on their dedicated machines?

My thought is I’d like a dedicated machine to spin up and develop stuff on for a personal project when needed but still have access to my experimental notebooks that I end up running here and there throughout the course.

Thanks!

1 Like

I just published a setup guide for AWS here

3 Likes

I just published a setup guide for AWS here. It uses a SageMaker Notebook Instance. Will hopefully get a SageMaker Studio version working soon…

2 Likes

Happy to help out! Just added a SageMaker setup guide :slight_smile:

2 Likes

This is great, thanks!
I managed to get a Sagemaker notebook instance to work with fastai2 following @pinaki’s instructions.
Having Studio too would be awesome.
I investigated it quickly yesterday but I was put away by this page.
Specifically this part


Maybe I am missing something super basic in the UI, but I could not find a way to set the instance type.
It apparently defaults to ml.t3.medium.
Super weird…

Awesome Matt - I’m so happy to see you here! :slight_smile:

I’ll be experimenting with this over the next couple of days as well. Any findings?

@bibsian @britt Yes! You can access /storage from Core. This directory is automatically mounted to your Linux (and Windows) machines in Core. Here’s a doc that covers this.

Feel free to ping me with any questions.

2 Likes

Thats awesome, thanks!

Vijay, I think the only limit would be the amount of RAM you have. 4GB may not be enough, but I would least try it. I know in the past friends were able to use their nvidia laptop gpu for training. Did you try?

This is so you only use notebooks to start your training job (you are not actually training in the notebook instance). If you do want to train on that instance, Amazon SageMaker > Notebook instances > YOUR NOTEBOOK > Edit notebook instance to change type. Your notebook has to be stopped to do this.

Hi @gamino thanks for the reply!
I am aware of this option for the standard SageMaker setup.
I was referring to SageMaker Studio, which @matt.mcclean was starting to investigate.
I could not figure out how to set the instance type into any notebook from the Studio IDE.
Hopefully I am missing something really basic.

1 Like

I actually haven’t tried Studio, but it looks cool. I would be interested to know if you get it to work. Sagemaker has come a long way!

1 Like