Anyone else see progress bar slow down massively sometimes? Why?

I use a tqdm progress bar on most long running tasks. I have noticed sometimes that it will fit 70% of the data for a model in 30 seconds; then the progress bar slows down and shows completion of the next 30% in 20 minutes.

I have tried removing all surplus code and not keeping the return predictions from fitting. Yet it still slows down. Memory shows plenty free. If I run the cell again it has identical behaviour, slowing at exactly the same point. More strangely the problem apparently cures itself eventually and it all starts working again at normal speed completing 100% in a minute.

Is this a memory thing? Or what? How can it be cured?