Developer chat

fastai fastai dev
750

BREAKING CHANGE

language_model_learner and text_classifier_learner have both been changed to look like create_cnn. Why? There are now more language model architectures in fastai, and hopefully soon, more pretrained models. So you should now use:

learn = language_model_learner(data_lm, AWD_LSTM)

for the old behavior of language_model_learner and

learn = text_classifier_learner(data_clas, AWD_LSTM)

for the old behavior of text_classifier_learner (see the test example).

The good thing is you can type Transformer or TransformerXL instead of AWD_LSTM and it just works :slight_smile: Well almost, you have too add pretrained=False because there are no pretrained models for those yet. You can still pass drop_mult in both cases, and if you want to modify the defaults, you can pass a config dictionary with all the values of the kwargs (the default configs are in the dictionaries awd_lstm_lm_config, awd_lstm_clas_config, transformer_lm_config,transformer_clas_config, transformer_XL_config, transformer_XL_config). There is an example of how to change their values in the tests.

The bad thing is that you won’t be able to load directly your old models for classification (language models are fine). I had to add another layer to make it work across architectures. Normally, this function should allow you to load an old model in a new learner:

def load_old_to_new(learn:Learner, path_to_model:PathOrStr):
    wgts = torch.load(path_to_model)
    if 'model' in wgts: wgts = wgts['model']
    wgts0 = OrderedDict({k[2:]:v for k,v in wgts['model'].items() if k.startswith('0.')})
    wgts1 = OrderedDict({k[2:]:v for k,v in wgts['model'].items() if k.startswith('1.')})
    learn.model[0].module.load_state_dict(wgts0)
    learn.model[1].load_state_dict(wgts1)
13 Likes