Datablock/Dataloader not working on SageMaker instance

Part 1 (2020)
1

I have a folder structure like this:

- notebook.ipynb
- data/
  - normal/
    - A/
    - B/
    - C/
    ...

Dataset is an image dataset with alphabet as labels (A,B,C,…) and I’m trying to create an image classification of letters (sign language to be exact). I’ve set up the Datablock using these

signlanguage = DataBlock(
    blocks = (ImageBlock, CategoryBlock),
    get_items = get_image_files,
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    get_y = parent_label,
    item_tfms=Resize(128))

So when I call signlanguage.dataloaders(“data/normal”), I get the “Could not do one pass in your dataloader, there is an something wrong with it” error. Data block seems to be set up correctly so I’m not sure what’s going on here. Can anyone help troubleshoot?

2

Could you show us the summary of the Datablock? here is the documentation

3

perhaps “data/normal” is not valid path to your directory

check if it will return array of paths to your directories (A, B, C etc.)

Path('data/normal').ls()

you may have to pass absolute path to your dataset

4

CUDA memory is running out. does that mean I need a bigger instance?

This returns the paths as expected.

5

Actually I had some of the lecture notebooks running. So when I shut all of them down, it worked as expected! I did request limit increase on a p2.8xlarge so hopefully I won’t have this issue in the future.