Suggestion about RNNLearner

fastai fastai dev
1

Hi, I found that RNNLeaner.language_model takes pretrained_fnames as input while RNNLearner.classifier doesn’t.
Also, load_pretrained invokes model.load_state_dict in strict mode.

So my suggestions are

  • support pretrained_fnames in RNNLearner.classifier
  • call model.load_state_dict with strict=False

Thanks beforehand

2

model.load_state_dict is always called on strict mode throughout the library, so I don’t see why we should change it there.
Also I’m unsure why we would want to directly pass the pretrained weights without the fine-tuning step before. That doesn’t sound like good practice in general. With an RNNLearner.classifier you are supposed to load the encoder from a previously fine-tuned model, so maybe adding an argument pretrained_encoder would be more interesting.

3

Maybe I misunderstood thnigs.
What I tried to do was load weights from finetuned saved LM model files.

4

In this case you want to use learn.load_encoder(name) where name has been previously saved with learn.save_encoder(name).
The docs have a fully working example.

1 Like