always running on CUDA GPU 0

Is there a way to control what GPUs uses memory from in multi-GPU machines?

What I’m seeing is completely ignoring os.environ["CUDA_VISIBLE_DEVICES"] = "3" to run on GPU 0, and only partially obeying torch.cuda.set_device(3), taking about 1 GB of memory on GPU 0 despite supposedly only being allowed to run on GPU 3 where it uses about 8 GB out of the 32 GB available. Is it a problem with CUDA version 11?

To compliment matters it seems a colleague of mine who’s also running on the same CUDA version doesn’t have this problem.

Ok. The solution is that os.environ["CUDA_VISIBLE_DEVICES"] needs to be set before is imported.

1 Like