How does slice behave for float inputs?

Part 1 (2019)
1

after finding a learning rate using lr_find and further training, a float(lr) is passed into the slice object constructor.
Searched quite a bit but haven’t found how the slice constructor behaves for float inputs.
need some insight as to how are we using it in the below statement

learn.fit_one_cycle(5,slice(lr))

also want to know how it works for 2 floats as inputs, for instance:
slice(1e-5,5e-2)

references are appreciated,
thanks in advance

2

From what I understood through reading the docs, there are two behaviours of slice:

  • When you only pass 1 argument (like the learningrate lr in your example), the last layergroups learningrate is lr and all others have lr/10
  • When you pass 2 arguments (like 1e-5,5e-2), the first group of layers gets 1e-5 as learningrate and the last group 5e-2. All other groups get a learningrate evenly geomatrically spaced between those arguments.

You can look it up in the docs: discriminative layer training and when you scroll down a bit to lr_range

Hope this helps :slight_smile:

1 Like