I’m working with the Stanford Dogs 120 dataset, and have noticed that I get the following pattern with ResNet-50 and ResNet-101 where in the second epoch the training and validation loss increases followed by the training and validation loss decreasing in the following epochs.
I am using
lr_find() to select a learning rate where the slope is steepest, and have experimented with different weight
wd and dropout
ps, but the pattern still happens. I’m wondering if this is normal or if it means there’s a setting I should change?