I would like to use VGG16 pretrained model but replacing the maxpooling at the end of the conv layer with a Flatten so it is comparable to the keras model.
I tried xtra_cut=3 and custom_head=myhead. However this removes the last 3 conv layers and leaves the maxpool layers in place.
Is it possible to load the pretrained weights and then remove some layers?
Partly found the answer. The xtra_cut removes layers from the first block. The pooling is in the second block.
Using xtra_cut=0 and custom_head fails with an index error (don’t know why).
Using xtra_cut=1 and then add back the removed layer in the custom_head it now creates the learner and looks OK…however now when I try the precompute it says “ValueError: array trailing dimensions do not match with self”.
I am sure there must be a really simple way to do this.
I don’t think, there is a simple way
nf = 512*14*14
data = ImageClassifierData.from_paths(PATH, tfms=tfms_from_model(arch, sz))
learn = ConvLearner.pretrained(
precompute=False, # Disable precompute
xtra_cut=1, # Remove max pooling
custom_head=nn.Sequential(nn.Linear(nf,2)) # Whatever
learn.models.nf = nf
learn.set_data(data,True) # Precompute
Getting closer but still doesn’t work. If I look at learn.models.top_model the Flatten is there. But if I look at learn.model it is missing. And on the set_data it fails with size mismatch.
Got it working without precompute. Can confirm that vgg16 with flatten and 256/128 dense layers is massively better than the maxpool version in fastai. I suspect it is because I have small images. I wonder if fastai could be adapted to be more flexible?
You can actually do this using nn.Sequential quite easily. Have a look at my notebook: https://github.com/TheShadow29/FAI-notes/blob/master/notebooks/Using-Pretrained-Pytorch-Models.ipynb.
Specifically, get the pretrained model using
md = learn.model. Print the model, keep the model till before the max pool. After that add a custom head. Suppose the max pool is at layer -5, you can do
new_md = nn.Sequential(*list(children(md))[:-5], custom_head). custom head basically includes everything you would need after the max pool. once you have the new model you can create a new learner using
learn_new = ConvLearner.from_model_data(new_md, data).
Hope this helps
Got the @bny6613 solution working. Precompute only works with pretrained models so have to use convlearner.pretrained. However it has to be called with precalculate=False so we can set the nf variable manually.
The reason it did not work before was I already had a fastai vgg16 model precalculated so the cache was present but different size. Working code is:
> def vgg16_keras(pre):
> model = vgg16(pre)
> model = nn.Sequential(*list(children(model)), Flatten())
> return model
> head = nn.Sequential(
> nn.Linear(512 * 1 * 28, 128),
> nn.Linear(128, 256),
> nn.Linear(256, 2)
> arch = vgg16_keras
> learn = ConvLearner.pretrained(arch, data, precompute=False, custom_head=head, opt_fn=optim.Adam)
> learn.models.nf = 14336
@simoneva You can use the precompute thing without pretrained as well, though that would need you to modify a bit of code. After you have created the learner object you could repeat the steps that you noted, and use
learn.save_fc1 I believe and set