Looking at this: https://gist.github.com/fchollet/7eb39b44eb9e16e59632d25fb3119975
# build a classifier model to put on top of the convolutional model
top_model = Sequential()
# note that it is necessary to start with a fully-trained
# classifier, including the top classifier,
# in order to successfully do fine-tuning
# add the model on top of the convolutional base
I am not sure I understand about loading top model weights - when it is expected to determine these weights by the process of fine tuning. Thoughts?
Keras uses a slightly different approach to fine-tuning to what I use - they save a separate weight file that doesn’t contain the last layer. That’s what they mean by ‘top_model’.
In that particular example, it looks like they are training 2 dense layers(?) (and not just last layer?). So I am guessing either you can preload weights for both these layers (which might give a better starting point for weights to optimize from and hence better model accuracy) or use random weights (might lead to less than optimal outcomes/accuracy)?
In that approach he is adding 2 dense layers (whereas we just add one), but training all the dense layers (see the ‘trainable=’ line).
I don’t think this is the best approach - I think training just the last layer before training more layers is a much faster and more reliable approach.