@gsg, See my post earlier where I started working on this - Developer chat - it got no feedback.
Also see Show GPU utilization metrics inside training loop (without subprocess call!)
and https://docs-dev.fast.ai/gpu#accessing-nvidia-gpu-info-programmatically.
I made a new mem-utils branch with the starter code. (new code is in fastai/utils/mem.py)
Also, this should work gracefully on CPU-only setups.