This seems to work fine to create the model, but I’ve been thinking: wouldn’t it make sense to only unfreeze this layer (full of zeroes) first and train it a bit, before unfreezing the entire model?
I am trying to figure how to unfreeze only the first layer of a resnet (rather than the last), not sure since even when the learner is frozen, I get the following:
first_layer = list(model.parameters())[0]
first_layer.requires_grad // returns True