Using plot_lr_find() with suggestions param

fastai
1

Hi,

When trying to use plot_lor_find() with the suggestions parameter as following -

lr_min, lr_steep, lr_valley, lr_slide = learn.lr_find(suggest_funcs=(minimum, steep, valley, slide))
print(f"Minimum/10: {lr_min:.2e}, steepest point: {lr_steep:.2e}, slide: {lr_slide:.2e}, valley: {lr_valley:.2e}")

fig = learn.recorder.plot_lr_find(return_fig=True, suggestions = (lr_min, lr_steep, lr_valley, lr_slide))
fig.savefig('lrfind.png')

I get the following error:

Traceback (most recent call last):
File “/NewRaidData/c16clf/ft_resnet34_fastai.py”, line 21, in
fig = learn.recorder.plot_lr_find(return_fig=True, suggestions = (lr_min, lr_steep, lr_valley, lr_slide))
File “/home/mnouyed/.local/lib/python3.10/site-packages/fastai/callback/schedule.py”, line 279, in plot_lr_find
for (val, idx), nm, color in zip(suggestions, nms, colors):
TypeError: ‘NoneType’ object is not iterable

It seems somehow the value of suggestions variable inside the if condition is getting out of scope.

https://github.com/fastai/fastai/blob/master/fastai/callback/schedule.py#L279

Could you please suggest a solution?

Thanks.

2

Hi,
I suggest you use the below code as an alternative.

lr_min, lr_steep, lr_valley, lr_slide = learn.lr_find(suggest_funcs=(minimum, steep, valley, slide))
plt.savefig('lrfind.png') # This does the saving as intended.

print(f"Minimum/10: {lr_min:.2e}, steepest point: {lr_steep:.2e}, slide: {lr_slide:.2e}, valley: {lr_valley:.2e}")

Now referring to your code everything seems logical but the following are to be noted:

  • The above error is due to the default value of nms being None. This variable actually collects the labels for the indicators for display in the legend. Thus, adding the following argument would solve that error;
 nms=('minimum', 'steep', 'valley', 'slide')
  • Doing this only generates another error since values of suggestions are to be unpacked. As in
        for (val, idx), nm, color in zip(suggestions, nms, colors):

is the code at the documentation .
Basically, each item in suggestions is expected to have the x and y coordinates of the respective indicator.

  • learn.recorder.plot_lr_find(return_fig = True) does not actually return anything or in other words returns None.
    Thus, fig.savefig('lrfind.png') would raise error;
AttributeError: 'NoneType' object has no attribute 'savefig'

NOTE: This can be solved by adding

return fig

as in,

def plot_lr_find(self:Recorder, skip_end=5, return_fig=True, suggestions=None, nms=None, **kwargs):
    "Plot the result of an LR Finder test (won't work if you didn't do `learn.lr_find()` before)"
    lrs    = self.lrs    if skip_end==0 else self.lrs   [:-skip_end]
    losses = self.losses if skip_end==0 else self.losses[:-skip_end]
    fig, ax = plt.subplots(1,1)
    ax.plot(lrs, losses)
    ax.set_ylabel("Loss")
    ax.set_xlabel("Learning Rate")
    ax.set_xscale('log')
    if suggestions:
        colors = plt.rcParams['axes.prop_cycle'].by_key()['color'][1:]
        for (val, idx), nm, color in zip(suggestions, nms, colors):
            ax.plot(val, idx, 'o', label=nm, c=color)
        ax.legend(loc='best')
    return fig

FINALLY, the code workaround all these is to run;


lr_min, lr_steep, lr_valley, lr_slide = learn.lr_find(suggest_funcs=(minimum, steep, valley, slide))
print(f"Minimum/10: {lr_min:.2e}, steepest point: {lr_steep:.2e}, slide: {lr_slide:.2e}, valley: {lr_valley:.2e}")

argg = (tensor(learn.recorder.lrs),tensor(learn.recorder.losses),len(learn.recorder.lrs))
fig = learn.recorder.plot_lr_find(return_fig=True, suggestions = [minimum(*argg)[1],steep(*argg)[1],valley(*argg)[1],slide(*argg)[1],
      ], nms=('minimum', 'steep', 'valley', 'slide'))
plt.savefig('lrfind.png')

All these references are found in https://github.com/fastai/fastai/blob/master/fastai/callback/schedule.py

I hope this answers you questions accordingly

1 Like