Yes, you can always manually stop the notebook instance as per the docs here
@matt.mcclean , I deleted the notebook and the config and started again, now it is fine,
i am suspecting that opening the notebook too soon is detrimental. on closer inspection the cloudWatch did not log anything on that first create. (it did after the delete and relaunch of the stack)
on a side note, changing ‘html5’ to ‘jshtml’ in the sgd notegbook is MUCH nicer and does not require any other installations. (and you can step one frame at a time.)
1 Like
All,
I followed the instructions here on how to install the SageMaker Notebook Instances. However, the fastai folder, with all the lessons was not install/appear. The only folder in the Jupyter Notebook was “Lost + Found”. Does anyone any a suggestion for getting the fastai folder with all the files to work? Thanks!
@matt.mcclean I had an issue re-starting the notebook instance. I checked the logs in Cloud Watch and it was caused by a conflict with git because I forgot and changed the notebook from the course. Therefore, I need to go to the instance terminal an run git stash. However, I was not able to initiate the instance, so I cannot launch the terminal. I went to the EC2 page in the console, and I cannot find the Sagemaker instance there, even though I uploaded ssh keys thinking about possibly accessing it from the terminal.
Another thought I had was to go to CloudFormation and temporarily change the config file to add a git stash line. However, I was not able to make it work either. At this point, I’m just going to install another sagemaker instance from scratch, but it’d be great to get some pointers about how to resolve the git conflict in case it happens again, if you could help. I’m not too worried about losing my notebooks as in addition to EBS I have also been downloading them locally.
Hi Paula. Make sure your custom script does not have a set -e at the beginning otherwise an error in running this script will give a failure message when starting the notebook instance.
If you are using SageMaker notebook instances, you will not be able to see them in the EC2 console. You need to go to the SageMaker web console and then goto the Notebook instances on the left navigation bar. You should see your notebook instance in the list there
It can take up to 10 minutes to install the course notebooks as the notebook will install the fastai library and dependencies in a background process
yup, you just need to wait a little longer and all the content will be available. It actually took a little more than 10 additional minutes to load everything the second time I set up the notebook instance, there are more materials in the repo, too.
I was not able to restart the notebook instance in the Sagemaker console, I attached the CloudWatch log bit with the error:
Since the instance was not initiated, I had no access to the command line, and could not fix the git conflict. The only option would be to edit the config file, but that did not work. I decided to not spend too much time on that and just setup another instance, which was much quicker this time (no need to wait for AWS approvals).
It’s been working fine with the second instance, I have stopped and restarted it with no problem.
thanks!
thanks!
I’ll try it again and wait
Setup reminders to start/stop SageMaker notebook instances
I have a created a demo project to send reminders on a fixed schedule to start and stop your notebook instances. For example, you can have it remind you to shut down your SageMaker notebook instance at 5pm every week day. It uses the AWS Step Functions service to orchestrate the workflow and allows you to start/stop instances via email. The project is setup using the new AWS Cloud Development Kit.
For details and installation instructions checkout the project here: https://github.com/aws-samples/cdk-sagemaker-notebook-workflow
3 Likes
Not sure if this is just me but I was having trouble installing isoweek for Lesson 6 - rossman. Running: !pip install isoweek in a cell installed it but then import isoweek would fail with a ModuleNotFound error. It’s like pip is not running in the fastai environment??
To fix this I had to go out to a terminal and do:
cd SageMaker
source activate envs/fastai
pip install isoweek
Install instructions are giving a 404 now.
Please try again. It was because it has moved to another repo here: https://github.com/aws-samples/cdk-sagemaker-notebook-workflow
Thanks! Also somewhat related, did the repo for the sagemaker fast ai container move as well (https://github.com/mattmcclean/sagemaker-fastai-container)? I have been messing around with best way to develop and set models up to train in a recurring batch and was basing some of my work off of it.
Yes, am moving this to a new GitHub repo. Will let you know when ready
Thanks a ton! Your repos and talk at AWS summit London have been a big help.
No worries 
I have published the new repo here: https://github.com/aws-samples/amazon-sagemaker-container-with-fastai which has been refactored to reuse the code from the standard Amazon SageMaker PyTorch container
Has something changed in Sagemaker recently to break this for fastai? I completely built a fresh notebook instance according to the instructions (because I was having problems) and now it seems that no fastai env is created:
sh-4.2$ source activate envs/fastai
Not a conda environment: /home/ec2-user/SageMaker/envs/fastai
envs/fastai doesn’t exist.