Any v2 callback implementations for Optuna or Ray Tune out there?

Looking to run some experiments with my blurr library with various:

  • epochs
  • LRs
  • schedulers / warmup ratios
  • batch sizes
  • Adam beta and epsilon params
  • dropout
  • weight decay
  • gradient clipping
  • etc…

Would love to see if there is a nice approach using one of the hyperparameter tuners along with Weights and Biases to do this.

Thoughts? Advice? Hit me up.


1 Like