Hello Fastai crew!
First post here…
Context - notebook 06 - multicat
Created a dataset consisting of one csv, plus train and validate folders with their respective pictures.
The csv file I created has the same format as in the exercise. However, when I train via
learn.fine_tune(), the error message
FileNotFoundError: [Errno 2] No such file or directory: '/content/gdrive/MyDrive/dest/dataset11/**train**/a-020_label_85.jpg
However, this file is in reality in the validation set.
Code for the data loaders:
dblock = DataBlock(blocks=(ImageBlock, MultiCategoryBlock),
item_tfms = RandomResizedCrop(128, min_scale=0.35))
dls = dblock.dataloaders(df)
And the other relevant functions:
def get_x( r ): return path/'train'/r['fname']
def get_y( r ): return r['labels'].split(' ')
`train = df.index[df['is_valid'] == 0].tolist()`
`valid = df.index[df['is_valid'] == 1].tolist()`
Can you share some insight as to what might be causing the confusion between training and validation sets?
thanks in advance!
In the muticat notebook, for the PASCAL_2007 dataset, the folders are structured like so:
And for setting up the DataBlock, we used everything in the
train/ folder and split the data set based on the accompanying
is_valid column. In your case though, you have two separate folders for
get_x() has to be updated somehow to return the full path, including the subdirectory path. Maybe you can prepend to
train.csv, the parent directory with
apply and try my previous answer.
Maybe something like this
p = lambda x: path/'valid/' if x else path/'train/'
df['fname_full'] = df['is_valid'].apply(p)/df['fname']
Now the column
fname_full has the full path to the file.
I’m not sure, but with what you have provided, it looks like you want to return
return r['fname'] in
Thanks @thatgeeman, that was it… I thought there were
train/ valid/ test/ directories, rather than just
Therefore, I see I have two options
a) leave my df as is fine, and copy the iimage files from
train/ and use the original code, or
b) change the df as you suggest, and keep the folders as is.
Thanks for the great in-depth answer!