Paperspace: Jupyter notebook server setup

[marvin (Marvin Hansen)]

@alessa,

yes setting up AWS takes a bit of hazzle. However, I did it today and running Week 1 notebook on a Volta, which is totally nuts because everything flies. However, there is concurrency bug that occasionally slows things down, so you can circumvent it by preventing subprocess spanning:

Very slow loading of the Convnet pretrained model on lesson 1

Ok, so we are using torch’s torch.utils.data.DataLoader which takes a parameter called num_workers to fetch data with multiple sub processes. num_workers (int, optional) – how many subprocesses to use for data loading. 0 means that the data will be loaded in the main process (default: 0) (defaut set in fastai lib is 4) The stalling is happening when we try to get the next batch iterator from the DataLoader fastai/dataset.py line 218 215 def __next__(self): 216 if self.i>=len(sel…