As we work on setting up our environments, I found this quite useful:
To check that torch is using a GPU:
In : import torch
In : torch.cuda.current_device()
In : torch.cuda.device(0)
Out: <torch.cuda.device at 0x7f2132913c50>
In : torch.cuda.device_count()
In : torch.cuda.get_device_name(0)
Out: 'Tesla K80'
To check that keras is using a GPU:
import tensorflow as tf
and check the jupyter logs for device info.
One could argue that ‘seeing’ a GPU is not really telling us that it is being used in training, but I think that here this is equivalent. Once a library sees the GPU, we are all set.
That is an interesting question and would be interesting to research. I do not have an answer myself.
I use nvidia-smi -l to see what my GPU is up to but this only gives you basic information. I think that newer versions of keras started preallocating all the GPU memory, so it doesn’t tell you much, but with torch you can see how much GPU memory is utilized by your process, that can be quite handy.
There are also more considerations that go into the batch size, so making it a heuristic to always stick as many training examples into a batch might not be ideal. Still, I think for this purpose - to have a cursory glance at model size in GPU given batch size and other parameters - nvida-smi seems to work quite nicely for torch at least based on my 2 hrs or so of experience with torch thus far
Yes that was a refreshing change with PyTorch. Keras (and I think it is tinkering down from Tensorflow backend) would grab the entire GPU memory by default and you would not know what is the actual usage.
Also, Tensorflow’s default setting is allocating full GPU memory during the run. Hence, with Keras and TF as backend, you’re most likely to see ~100% memory being allocated. This default behavior can be changed by @jeremy’s tip here (Tip: Clear tensorflow GPU memory).
PyTorch is nicer on this aspect. Allocating just enough memory as it needs.