That makes sense…
And that’s correct, I hadn’t yet unfreezed the model. After you mentioned this, however… I did unfreeze and ran a few experiments. Unfortunately, I think I ran into overfitting after running ±250 epochs. I think I might start fresh and try to find a higher learning rate that works or bump the images up to 64x64. If this doesn’t work I guess I’ll have to dig a little deeper into network architectures and the list from @abhikjha. I haven’t yet reached a lecture that discusses overfitting in detail, but I’m guessing that will come later in the course.
Is there a good rule of thumb on when is an appropriate time to unfreeze? After seeing a certain loss value? After a certain number of epochs with continual loss decrease? etc?
