after finding a learning rate using lr_find and further training, a float(lr) is passed into the slice object constructor.
Searched quite a bit but haven’t found how the slice constructor behaves for float inputs.
need some insight as to how are we using it in the below statement
also want to know how it works for 2 floats as inputs, for instance:
references are appreciated,
thanks in advance
From what I understood through reading the docs, there are two behaviours of
- When you only pass 1 argument (like the learningrate
lr in your example), the last layergroups learningrate is
lr and all others have
- When you pass 2 arguments (like
1e-5,5e-2), the first group of layers gets
1e-5 as learningrate and the last group
5e-2. All other groups get a learningrate evenly geomatrically spaced between those arguments.
You can look it up in the docs: discriminative layer training and when you scroll down a bit to lr_range
Hope this helps