Looking to run some experiments with my blurr library with various:
- epochs
- LRs
- schedulers / warmup ratios
- batch sizes
- Adam beta and epsilon params
- dropout
- weight decay
- gradient clipping
- etc…
Would love to see if there is a nice approach using one of the hyperparameter tuners along with Weights and Biases to do this.
Thoughts? Advice? Hit me up.
Thanks!