I am new to deep learning and working on SIIM-ISIC Melanoma Classification problem statement on kaggle. Training data consists of around 30K jpeg images with space size ~50GB.
While I am able to build a simple model,I noticed that GPU utilization is zero although it’s occupying some memory( around 1GB) .As expected it’s taking me lot of time to train because of this.
I noticed that when i reduced batch_size to 4 from 32, GPU utilization reached 4% repeatedly for very short intervals . why is this happening? Is there any relationship with batch_size and GPU utilization ,if so how should i optimally set batch_size so that i can utilize GPU effectively.
The larger the batch, the more effectively the GPU can run. If the batch size is very small, there is relatively a lot of overhead in firing up the GPU and waiting for the results. With a larger batch size, that overhead still exists but is now amortized (divided) over more examples and so you spend less time waiting. Ideally, your GPU should be 100% busy when training.
However… if you do a lot of data loading and preprocessing on the CPU, and you have a slow CPU or a slow disk, then the GPU will have to wait on the CPU. If this happens, you’ll notice that your CPU is 100% busy while the GPU is doing almost nothing. So also look at what the CPU is doing.
If both the CPU and GPU do not seem to be very busy, check the I/O rate of your disk. If you’re using a spinning hard disk instead of an SSD, that might be the speed bottleneck.