I setup this computer to use remotely. In some instances CUDA errors (maybe related to network issues, I can’t tell) left the GPU useless. Killing the jupyter kernel didn’t help, only a computer restart.
This is the only GPU in the system (1070ti), so I believe it’s in use by the display. I am not running xwindows or similar.
nvidia-smi reset doesn’t seem to help either:
tbatchelli@MLrig:~$ nvidia-smi -r GPU Reset couldn't run because GPU 00000000:23:00.0 is the primary GPU.
Is there any way to reset the GPU without restarting linux? Would adding a second (cheaper) GPU for display help?