Lesson 1: Choosing a learning rate doesn't work on my dataset

Part 1 (2018)
1

After the explanation of the choosing a learning rate part, I changed my path to my own dataset, and re-ran the cells using that data. I expected to see something similar to the dogs vs cats dataset, but I didn’t:

  • The lr_find() was super quick, went to 100%, and only got to an accuracy of 0.6
  • The learn.sched.plot_lr() is a linear line from 0 at 1 iteration, to 0.01 at 2 iterations
  • The learn.sched.plot() has no line showing (axis are 10^0 to 10^1 learning rate, -0.04 to 0.04 loss)

I have a couple of thoughts why this might be:

  • My data sets are very small (~50 training samples each, ~15 validation samples each)
  • The accuracy I achieved previously on them is only ~72%

But I was wondering if somebody could help me understand a little more what’s going on and why this not-so-great dataset doesn’t work with the learning rate fitting code.

Thanks! :slight_smile:

2 Likes
2

What is your batch size? The LR finder works by incrementing the LR in every batch. But if you only have 50 training samples and your batch size is something like 32 then the LR finder only gets to try out two learning rates.

2 Likes
3

This was it, thankyou! The default batch size is 64, so yea… :slight_smile:
Concretely, in code:
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz), bs=4)

1 Like
4

Thank you for being explicit. I was not understanding the batch size concept until I read your post!