Learn.sched.plot() doesn't show anything


#1

Hello everyone,

I have the following problem: I tried to do the dog breed competition, and in order to be able to use my personal machine without waiting too much, I took only the first 1001 rows of the .csv table.
But when I try to use the learning rate finder, nothing happens. I don’t think that I am in the case explained here:


because I have no plots at all:

and because I’m supposed to have 800 images in the training set and a batch size of 20.
The output of learn.lr_find() is also strange as you can see.

I don’t know what may cause this error. Probably a stupid mistake from mine but I can’t find it.
And in case you wonder, I do use
%matplotlib inline

I have the feeling that somehow the NN doesn’t see the data, but I don’t see anything wrong in the way I define my neural network:


#2

I encountered the same issue. Inside the method that plots the learning rate, there is a piece that doesn’t work reliably inside a jupyter notebook. I think there is a merge request on github with a discussion on this.

Not sure what the status is and can’t check right now nor can’t look this up for you in code. The way I fixed this was by commenting out an if condition. If you installed fastai using pip this might be a bit more challenging but if you are just requiring the library that sits locally in a directory on your hard drive, then you can fix this by searching for wherever plot_lr is defined and messing with the if condition.

Ok, did a quick search:

    def plot_lr(self):
        if not in_ipynb():
            plt.switch_backend('agg')
        plt.xlabel("iterations")
        plt.ylabel("learning rate")
        plt.plot(self.iterations, self.lrs)
        if not in_ipynb():
            plt.savefig(os.path.join(self.save_path, 'lr_plot.png'))

This is whats in sgdr.py

Probably something like this should work (just remove the lines):

    def plot_lr(self):
        plt.xlabel("iterations")
        plt.ylabel("learning rate")
        plt.plot(self.iterations, self.lrs)

I am quite sure this will be fixed soon so maybe look up the discussion or github or just give it a couple of days :slight_smile: Such things are to be expected when you are on the cutting edge :wink:


#3

It worked!

Thanks a log Radek!


(Anders) #4

I also experienced getting an empty plot from learn.sched.plot(), so I tried modifying plot_lr in my /fastai/sgdr.py, but now I get this error:

AttributeError Traceback (most recent call last)
in ()
----> 1 learn.sched.plot()

AttributeError: ‘CosAnneal’ object has no attribute ‘plot’


#5

The method name for plotting the learning rate is plot_lr.

Might be you modified the file too much - best bet would be to restore it and only make controlled changes.


(Anders) #6

Thank you for the correction. I must be making multiple errors because now the plot function is working again (without restoring sgdr).
Edit:
I think my dataset is too small. If I reduce the batch size to 2 then I can get a plot.
I downloaded a horses vs donkeys dataset from kaggle and I might need a better distribution between test/train/valid. I put 267 images in the validation set.

The question I should be asking:
What should batch size be in relation to dataset size?


#7

Ah sorry - seems the correct way of looking for the right lr is to first do learn.lr_find() and then learn.sched.plot(). Sorry for the earlier incorrect info.

Horses vs donkeys sounds very neat :slight_smile: Good luck with your project!


(Jaggo Re) #8

Thanks, this helped.


(Francisco José Tornay Mejías) #9

I had the same problem and I followed the advice on this thread but I still get a blank plot


Using a CNN to Tell Tiffany and Ivanka Trump Apart
(Bharat Kunwar) #10

I do not have a lot of training images ~ around 400 images per class. I had a blank plot but I tried decreasing the batch size from whatever the default value was to 8 like this:

bs = 8
ImageClassifierData.from_paths(PATH, bs=bs, tfms=tfms_from_model(resnet34, sz))

And viola… I finally had a learning schedule plot!


#11

Sometimes it can help to give the learner a range to look at, for instance:
learn.lr_find(start_lr=1e-5, end_lr=1e-1)


(Radi) #12

Having the same issue, and the solutions didn’t help.

What should I try next?