Using Large batch_size for DataLoader?

[mcclomitz (Kieran)]

Hey Nyx - I was having similar questions recently and found some good insight from the Leslie Smith hyper-parameters paper.

You can find a link to the paper and some discussion around it here:

[Paper discuss] A disciplined approach to neural network hyper parameters by Leslie Smith

I am reading the paper by Leslie Smith https://arxiv.org/abs/1803.09820. Question#1 In the paper Leslie mentions … the test loss within the black box indicates signs of overfitting at learning rates … [image] I understand that as the training loss is less than test loss it means we are overfitting. But isn’t that true for the whole graph? I mean the test loss is more than train loss everywhere. So why does this box indicates signs of overfitting? Question#2 Leslie shows this picture in …

Best of luck