NEW beta learning schedule
The TWiML Deep learning From the Foundations Study Group repo now has an updated version of 05_anneal_jcat.ipynb that adds a beta learning schedule.
The beta probability distribution function (pdf) is defined on the interval [0,1], and has a varying shape that is controlled by its two input parameters a and b.
When a and b are both greater than 1, the beta pdf is positive and convex on the interval [0, 1] with zeros only at the endpoints, allowing the construction of an infinite variety of smoothly continuous learning rate schedules that start and end with a learning rate near zero.
See the purple curve in the plot below.
You can adjust the shape of the beta pdf schedule by varying the values of the parameters a and b.
If
- a = b
- a\lt b
- b\lt a
the beta pdf is concentrated
- symmetrically
- in the left half
- in the right half
on the interval [0,1]
.
