How to set cycle_len with callbacks?

Looking at this code here:

on_end = lambda sched, cycle: save_model(m, f'{PATH}models/cyc_{cycle}')
cb = [CosAnneal(lo, len(md.trn_dl), cycle_mult=2)]#, on_cycle_end=on_end)]

fit(m, md, 2**4-1, lo.opt, F.nll_loss, callbacks=cb)

… and trying to figure out how to set cycle_len, so for example, I can run 3 cycles each composed of 3 epochs.


I think (without testing) it’s:

cb = [CosAnneal(lo, len(md.trn_dl)*3)]
fit(m, md, 9, lo.opt, F.nll_loss, callbacks=cb)
1 Like

I tried that. The code runs but my model doesn’t improve as expected (based on what I see when I fit with the cycle_mult=2 where longer cycles show better results).

Is there a way to check that it is indeed doing SGDR (using 3 epochs per cycle) rather than just treating each of the 9 epochs as separate cycle?

Yup cb[0].plot_lr

BTW, just got around to testing this and you are correct.

Love being able to access so much of the goodness through the callback instance, I had no idea what was available.

1 Like