I’m running lesson 1 with my own example data: Frogs vs Toads. I have 10 images of each in various dimensions in *.jpg format. My folder structure matches the dogscats structure namely:
frogstoads/
models/ (empty)
sample/ (empty)
test1/ (empty)
train/ (empty)
valid/
frogs/
toads/
I’ve changed the paths to match mine in the notebook, but it throws an error when I’m instantiating the image classifier. This line:
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz))
Throws this error:
ValueError: zero-size array to reduction operation maximum which has no identity
I commented out the learn and learn.fit(...) lines. Error image below:
Ok I got this working - it had no ‘doh’ training examples. So i copied the valid folder data i.e frogs and toads to the train folder.
This begs the question though, currently - the train and valid folders are identical. Should the valid folder only have a few examples of each (frogs and toads) in them?
The valid folder should have as many examples of frogs and toads as you have images that are not in your train folder. That is, all else being equal more validation images are better for evaluating your model but these images should be mutually exclusive with your training images.
cool - I tried again with a new data set comparing dooplegangers (two sets of similar faces). What I’m finding is that the smaller the set of data the larger the variance needs to exist in the images, and i guess in turn larger assumptions it needs to make. The larger the data set the smaller the variance can be and in turn smaller assumptions. Larger data sets also mean you can start of with large assumptions and refine those assumptions (through differing learning rates) to deal subtler variance.
I’ve got the same problem but with validation set even though I have good number of examples in my valid folder. I didn’t understand the error. could anyone explain what the error tells?
