Horrible results in lesson 2 (bears classification)

When classifying the bears I get nowhere near the results of Jeremy:

The first cycles get me a error_rate around 10%. I tried changing the seed number, but it never gets close to 2% like in the video.

Trying to fit the learning rate does not help me much:

Compare with Jeremys graph which is much more distinct:

I have 200 downloaded images of each class.

Any ideas? Am I doing something wrong or is this a expected deviation because our datasets are not exactly the same?

Try deleting your bears folders and try running the notebooks again. I remember the first time I tried running the notebooks I screwed up the data for the folders.

After correcting it, I was getting close to or even better than the notebooks.

Thanks J. D.!
It worked, but now it might have gotten too good. I cant see the learning rate graph… :rofl:

Anyways the results are good now!

Hi there,

I trained resnet50 on bear dataset and got the following result.

epoch train_loss valid_loss error_rate
1 0.109231 0.091223 0.032154
2 0.082727 0.113479 0.035370
3 0.094685 0.088089 0.032154
4 0.077283 0.090468 0.032154
5 0.052039 0.094755 0.028939
6 0.055198 0.109953 0.032154
7 0.059238 0.101610 0.028939

All good when I test with an image of black or grizzly or teddy bear. But, if I test on a random picture sometimes I get a very high probability. For example, if I try with a random picture, I get 80% grizzly. Can you please tell me what’s going on? Thanks in advance.

The model only knows how to recognize three classes. Teddy bears, grizzly bears, and black bears. When presented with an image that doesn’t correspond to any of these categories, it tries its best to give a good estimate of what the image is.

However, because the image doesn’t fit in with the other classes, the model doesn’t know how to classify it, hence the 80% grizzly guess.

@jdm, Thank you very much. I got it. I wasn’t thinking enough in the late night. :slight_smile:

No problem :slight_smile:

Hi JD, to clarify, do you mean delete all of the downloaded images/datasets that I currently have and re-create the datasets again? I’m also getting a higher error rate so going to try out your method

AFAIR,
You have to create 3 folders for 3 categories of bears. Next you have to make sure each image has only a type of bear.
If a image has two type of bears then, classifier will learn about the bear from the folder name.

To achieve more accuracy, can you check the data augmentation & image transformation.?

Yes, because if you are getting anything lower than what is normally expected (usually anywhere from 80-100%), and more like 40-50%; it is probably because your dataset is not set up properly. You can see if this is the problem by doing

data.show_batch(rows=3,  figsize=(6,6))

and seeing if your data is either mislabeled or improperly prepared. If it is, then you can try to delete & re-download all the images to see if the problem persists.