Edit: I’m not sure if this is the right forum to post this!
I’ve taken a stab at Jeremy’s idea this morning about autoRLfinder.
The output of LRfinder is a univariate time series, and as such can easily be transformed into an image, in a similar way to what I showed in a ‘Show your work’ post. The outcome of that experiment using transfer learning in a time series classification problem with only 30 train samples was really good, close to state of the art, and with only standard setting.
What I’ve done is the following:
- Created a synthetic dataset roughly mimicking lr_finder outputs (time series)
- Estimated a ‘recommended lr’ (this is a very, very bad approximation). I’ve just used to show the proposed methodology.
- Padded time series data as they don’t all have the same length.
- Transformed ts into images (this is based on a transformer that creates a rich representation of a univariate time series - paper), , for example:
- Created a regression problem where the target is the log of the lr.
- Train an model in the standard way (so far I have not been able to finalize this, as for some reason I keep getting an error. I’m still working on this)
The way I envision this working would be:
- Run the lr_finder with an option to recommend a learning rate
- Lr_finder would be run, and the model would predict a recommended lr when finished
I’m not too familiar with fastai library yet, and would need some help to get this working.
So if anybody find this interesting, and would like to collaborate, this is what would be required to complete the project:
- Create a dataset with lr_finder curve values and ground truths.
- Train the regression model
- If the result is good, integrate it into the fastai code
I’ve created a gist with the work I’ve done so far.
I’ll be happy to hear:
- If this is of interest to you
- Feedback, ideas, recommendations on how to get what’s still missing