In the case of transfer learning there are two phases. First with the pretrained part frozen and only the newly added parameters are trained (by default for 1 epoch) and then the whole model together with the pretrained part. See the code here.
As for your general question, I’ve actually created a tutorial for that