Fast.ai on sagemaker, continues re-downloading of resnet34

fastai
1

hi,

Setting up fastai on the Sagemaker is a very smooth process and works well! Thank you!

I wonder why the following code results in re-download of resnet34 every time I restart the notebook? In’s storage persistent?

arch=resnet34
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz))
2

the sagemaker tmp directory is reset on restart, make sure your data path doesnt cache things there

3

Everything not under ~/SageMaker is not saved between notebook restarts. You need to remap the ~/.fastai directory to somewhere on ~/SageMaker. Some example scripts to run on notebook creation are:

mkdir /home/ec2-user/SageMaker/.torch && ln -s /home/ec2-user/SageMaker/.torch /home/ec2-user/.torch
mkdir /home/ec2-user/SageMaker/.fastai && ln -s /home/ec2-user/SageMaker/.fastai /home/ec2-user/.fastai

Every time you start the notebook you will need to recreate the symlinks with the commands:

ln -s /home/ec2-user/SageMaker/.torch /home/ec2-user/.torch
ln -s /home/ec2-user/SageMaker/.fastai /home/ec2-user/.fastai

The SageMaker setup notes described here do this for you automatically.