wgpubs
(WG)
December 22, 2017, 11:13pm
1
Looking at this code here:
on_end = lambda sched, cycle: save_model(m, f'{PATH}models/cyc_{cycle}')
cb = [CosAnneal(lo, len(md.trn_dl), cycle_mult=2)]#, on_cycle_end=on_end)]
fit(m, md, 2**4-1, lo.opt, F.nll_loss, callbacks=cb)
… and trying to figure out how to set cycle_len
, so for example, I can run 3 cycles each composed of 3 epochs.
Thanks
jeremy
(Jeremy Howard)
December 25, 2017, 5:59am
2
wgpubs:
cb = [CosAnneal(lo, len(md.trn_dl), cycle_mult=2)]#, on_cycle_end=on_end)]
fit(m, md, 2**4-1, lo.opt, F.nll_loss, callbacks=cb)
I think (without testing) it’s:
cb = [CosAnneal(lo, len(md.trn_dl)*3)]
fit(m, md, 9, lo.opt, F.nll_loss, callbacks=cb)
1 Like
wgpubs
(WG)
December 31, 2017, 8:42pm
3
I tried that. The code runs but my model doesn’t improve as expected (based on what I see when I fit with the cycle_mult=2 where longer cycles show better results).
Is there a way to check that it is indeed doing SGDR (using 3 epochs per cycle) rather than just treating each of the 9 epochs as separate cycle?
wgpubs
(WG)
March 16, 2018, 5:14pm
5
BTW, just got around to testing this and you are correct.
Love being able to access so much of the fast.ai goodness through the callback instance, I had no idea what was available.
1 Like