Examples from Keras Documentation

Part 1 (2017)
#1

Looking at this: https://gist.github.com/fchollet/7eb39b44eb9e16e59632d25fb3119975

# build a classifier model to put on top of the convolutional model
   
        top_model = Sequential()      
        top_model.add(Flatten(input_shape=model.output_shape[1:]))
        top_model.add(Dense(256, activation='relu'))
        top_model.add(Dropout(0.5))
        top_model.add(Dense(1, activation='sigmoid'))

        # note that it is necessary to start with a fully-trained      
        # classifier, including the top classifier,
        # in order to successfully do fine-tuning
        top_model.load_weights(top_model_weights_path)

       # add the model on top of the convolutional base
        model.add(top_model)

I am not sure I understand about loading top model weights - when it is expected to determine these weights by the process of fine tuning. Thoughts?

#2

Keras uses a slightly different approach to fine-tuning to what I use - they save a separate weight file that doesn’t contain the last layer. That’s what they mean by ‘top_model’.

#3

In that particular example, it looks like they are training 2 dense layers(?) (and not just last layer?). So I am guessing either you can preload weights for both these layers (which might give a better starting point for weights to optimize from and hence better model accuracy) or use random weights (might lead to less than optimal outcomes/accuracy)?

#4

In that approach he is adding 2 dense layers (whereas we just add one), but training all the dense layers (see the ‘trainable=’ line).

I don’t think this is the best approach - I think training just the last layer before training more layers is a much faster and more reliable approach.

1 Like