Resnet34 vs resnet50

I have an image classifier trained on resnet34, using image size 234 on my own image dataset. If I simply change the architecture to be resnet50 and increase the image size to be 300 instead just like in the course, I get worst performance (3% less accuracy). I have around 950k images… I would have thought that resnet50 having more parameters could take advantage of the larger size of the images.

What could cause a worst performance from resnet50?



I have the same issue and in my case it is not even on my own image dataset but the course standard image set from the lesson 1. I was thinking by using a larger image the accuracy will increase and the resnet50 will have deeper layers so there would be a significant improvement to the accuracy