Learn.recorder.plot - understanding of skip_start and skip_end

[partham]

Hi Jeremy/FastAI members,

First of thanks for such a great resource for learning. Presently I’m on the lesson 1 of part 1, and I’m trying modelling with my own data - I have chosen the FashionMNIST data set here. And I’m loading the pixel values as a data loader in torch and then passing it on to ImageDatabunch (of course, show_batch isn’t implemented there - so, that’s something I’m yet to figure out).

But, I’m finding it hard to run & visualize lr_find and recorder.plot.

So, I was getting a blank plot while trying to visualize - when I noticed it’s always showing me values between 1e0 and 1e1. So, I started experimenting with adding
start_lr/end_lr in lr_find and skip_start/skip_end in recorder.plot.

but, interestingly running the same code gives different results here:

I think I understand what start_lr and end_lr does - but, I’m not sure how the value of skip_start/skip_end influences the plot?

[partham]

Nevermind, apparently, not having proper loss function was the issue!

after creating cnn_learner, just check the learn.loss_func. Most likely it’s going to be nll_loss, change that to torch.nn.functional.cross_entropy

Source : negative loss with mnist

Post that it’s going to work fine. Although I’m still not sure about how skip_start works, will try out variations to understand better

[NathanHub (Nathan Hubens)]

Hi ! The skip_start and skip_end are there if you want to ‘skip’ some point in the plot (at the beginning or at the end respectively).

This can be useful when you loss goes very very high at the end, sometimes leading to a final graph that is ‘squished’, making it unreadable.

Take a look at this post where it happend: Sched -> recorder?

Hope that helps

[partham]

Hey @NathanHub! Thanks for taking the time out to comment. Your link was also helpful.

But, I think originally I wasn’t able to understand how skip_start chooses “some points to skip”?

If my start_lr is 1e-8, and skip_start is 1, does it mean it would skip 1 to the right (in log scale) - so, my map should plot from 1e-7 onward (for -1 then, the plot should be from 1e-9)?

Or is it more like the example in the link you shared - losses[skip_start,-skip_end] i.e. indices for the loss list?

[partham]

Btw, while looking at different problem, I sort of stumbled upon the query I had. So, from the code base it looks like the 2nd conjecture I had is the correct one. Here’s how skip_start and skip_end is implemented: