What the parameters in model.fit means?

[cedric (Cedric Chee)]

Hi. Yes, this is not so clear as the library lacks proper documentation. However, do look out for sgugger’s posts (articles acting as quick documentation) across the forum to learn how to use fastai API like mode.fit, etc. I think the following thread is a good starting point:

Using use_clr_beta and new plotting tools

I’ve written a few things for the fastai library while working on my project around Leslie Smith’s last article and I wanted to share them with you. use_clr_beta : This is the implementation of Leslie Smith’s 1cycle policy as detailed here. He recommends to use a cycle like this one: [art5_lr_schedule] for the learning rates during the training: equal time growing and descending, and a bit of space left to get the learning rate really slow. When used in fit, the value at the top (here 0.01)…

which course it mentioned.

Part 2 lesson 13 video, Jeremy did a demo on the new TrainingPhase API implemented by sgugger. Check out the notebook showing examples on how to use the Training Phase API together with 1cycle policy implemented in fastai library and available to the model.fit function as parameters like use_clr, cycle_len, etc.

I have written the following thread (outdated a little bit) if you want to learn more about 1cycle policy, super convergence and a simple guide on how to use these methods (part of fastai library) from our codes:

The 1cycle policy - an experiment that investigate super-convergence phenomenon described in Leslie Smith's research

This is an interesting experiment conducted by a fellow under fast.ai’s International Fellowship 2018 that dig into Leslie Smith’s work that Leslie describes the super-convergence phenomenon in this paper, “A Disciplined Approach to Neural Network Hyper-Parameters: Part 1 - Learning Rate, Batch Size, Momentum, and Weight Decay”. Results from the experiments: By training with high learning rates we can reach a model that gets 93% accuracy in 70 epochs which is less than 7k iterations (as opp…

Even more questions and examples around this topic:

Understanding use_clr

So at the end of the pascal notebook we have this: learn.fit(lr, 1, cycle_len=3, use_clr=(32,5)) Looking at the code for learner.py I found that when specifying use_clr we end up using a CircularLR schedule instead of a CosAnneal. But even after looking at the CircularLR code I’m having trouble identifying what the parameters clr_div, cut_div means, and why is better to use a Circular anneal instead of a Cosine in this case?