Typical Training process

Part 1 (2019)
1

I wonder if there is a typical path to follow when training a model. Is it always:

  • create learner
  • choose transfer model weights
  • run lr_find
  • train last layer
  • run unfreeze
  • train whole model

finished.

So the point if:

  • Do I always have to run unfreeze and train the whole model to improve results?
  • does it make sense to freeze again and to some more training on the last layer?
  • Do I always run unfreeze without parameters or should I try to unfreeze different numbers of layers? (unfreeze(-1) train -> unfreeze(-2) trainer, etc.
2

My answer to each of those questions is that yes, it makes sense to try different things out, and see if you get better results.
Then if some new technique seems to give better results consistently, share it with everyone else!

You have the right mindset to question everything.

1 Like