Understanding the use of learn.fit() multiple times in the lessons

In the lessons, I notice we often fit the model multiple times. For example in Lesson 3 - Rossmann, learn.fit is called three times (reproduced as follows):

m = md.get_learner(emb_szs, len(df.columns)-len(cat_vars),
                   0.04, 1, [1000,500], [0.001,0.01], y_range=y_range)
lr = 1e-3
m.fit(lr, 3, metrics=[exp_rmspe])                     # fit 1
m.fit(lr, 5, metrics=[exp_rmspe], cycle_len=1)        # fit 2
m.fit(lr, 2, metrics=[exp_rmspe], cycle_len=4)        # fit 3

My understanding from reading other posts is this trains the model incrementally. Could someone please explain the following to a beginner: (1) what does multiple fitting actually entail and why we do it in the first place, (2) what is considered good practice for multiple fitting, i.e., why does the above sequence of fits makes sense (as opposed to, say, fit 3 first, then 2 then 1), (3) is there a way to “rollback” a fit step, i.e., tell the model to revert to a previous state?

Thanks for reading and answering!

Each time you call .fit the model weights get updated (if you run model unfreeze before fit all weights are updated, if you don’t run unfreeze before fit only the weights in the last n layers are modified).

To rollback, you would need to first run learn.save(‘type-your-model-version’) after each learn.fit(). Then if you want to go back to a previous state you can run learn.load(‘themodelyouwanttoload’).

Regarding how many epochs and how many times you run .fit, there is no hard rule but 3-5 epochs per run seems pretty popular, and 1-3 calls of .fit also is common, it depends on your validation losses and whether your model is good enough-and you’ll get a feel for this by running lots of tests to see what happens.

2 Likes