Error while adding Test Folder

Part 1 (2019)
1

I am using a custom dataset with the following structure.

image
image.png1478×392 18.9 KB

I got an error(Wrong Databunch composition) while creating the ImageDataBunch object with data blocks API.

image
image.png1401×741 46.1 KB

As you might see in the image the test folder is also loaded in the training data set and is considered as a class.

This is an imagenet data structure, so ideally the images in test set should not go into the training dataset. Is this because of a library bug or am I missing something?

I tried inferencing using predict but since my test data set is very large, it takes a lot of time to run.

1 Like
2

For anyone still facing this issue.
I found a work around to it.

Try

data = (ImageItemList.from_folder(path/"train")  # Where to find the data? -> in path and its subfolders
    .random_split_by_pct()                       # How to split in train/valid? -> use the folders
    .label_from_folder() #How to label? ->       # depending on the folder of the filenames
    .add_test(ItemList.from_folder(path=path/"test-images",label=None))
    .transform(get_transforms(),size=224)        # Data augmentation? -> use tfms with a size of 64
    .databunch(bs=bs).normalize(imagenet_stats))
2 Likes
3

This worked !! Thanks. @rishabh

4

@rishabh This is is not working for me. I am getting error . Please suggest how to fix the issue
TypeError: unsupported operand type(s) for /: ‘str’ and ‘str’

image
image.png964×402 19.8 KB

5

@ritika26 I guess your path variable is of string type rather than PosixPath object.

Also check if you have the same folder structure as shown in original post.