Hilbert Constrained Gradient Descent

Now that are in Optimizers Hunting Season, had anyone tried Hilbert Constrained Gradient Descent? I had this paper sitting at my todo list for what feels like ages already.

The concept itself is intriguing, to say the least. Requires to change the training loop to feed a test set in the loop, and they say it is slow BUT converge fast faster (less iterations). That’s the reason why I havent tried it yet.

Paper: https://openreview.net/pdf?id=SkMwpiR9Y7
Code: https://github.com/KordingLab/hilbert-constrained-gradient-descent


Wow this paper is quite interesting! I’ll try to find some time to test it for sure.