Validation Data accuracy/loss calc

Part 2 (2019)
1

3 things really, from lesson 9 v3 course. Part about calculating accuracy / loss for the validation set.

  1. In nv = len(valid_dl) what does nv stand for?
  2. How is it that the mini batch sizes are different? As far as I can see, only the very last minibatch will have a different size. Is that what we are trying to account for?
  3. Why do we double the batch size in the dataloader for validation as opposed to training. Just for speed?
2

hi @abcde13

  1. nv is just a variable name. Doesn’t matter. He probably mean ‘number (of) validation points’ or something.
  2. Yes, thats exactly what he is trying to account for. But the last batch itself may create a huge impact!
  3. Not exactly for speed. During training, not just the loss is calculated, but the gradients of the loss are calculated too. In validation, we don’t calculate gradients, so that leaves us with more space in the GPU/CPU. So we can now use double the batchsize during validation, as compared to training. The bigger the size of a batch, the more stable, generalized results you get.
1 Like
3

Ahh, thank you. This all makes sense