Loading model, that was saved with fastai1

fastai fastai dev
1

Hello everyone,

I am facing the following issue:
I have a saved model, that i once trained with fastai1.
The architecture is torchvision.models.resnet.resnext101_32x8d

Now I am trying to load the weights of that model to a learner in fastai2.
I am using the exact same architecture.
Unfortunately it is not working and I get the following error:

RuntimeError Traceback (most recent call last)
in
----> 1 learner.load(model_path)

/opt/conda/envs/DLM_Py3/lib/python3.7/site-packages/fastai2/learner.py in load(self, file, with_opt, device, **kwargs)
271 if self.opt is None: self.create_opt()
272 file = join_path_file(file, self.path/self.model_dir, ext=’.pth’)
–> 273 load_model(file, self.model, self.opt, device=device, **kwargs)
274 return self
275

/opt/conda/envs/DLM_Py3/lib/python3.7/site-packages/fastai2/learner.py in load_model(file, model, opt, with_opt, device, strict)
52 hasopt = set(state)=={‘model’, ‘opt’}
53 model_state = state[‘model’] if hasopt else state
—> 54 get_model(model).load_state_dict(model_state, strict=strict)
55 if hasopt and ifnone(with_opt,True):
56 try: opt.load_state_dict(state[‘opt’])

/opt/conda/envs/DLM_Py3/lib/python3.7/site-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict)
828 if len(error_msgs) > 0:
829 raise RuntimeError(‘Error(s) in loading state_dict for {}:\n\t{}’.format(
–> 830 self.class.name, “\n\t”.join(error_msgs)))
831 return _IncompatibleKeys(missing_keys, unexpected_keys)
832

RuntimeError: Error(s) in loading state_dict for Sequential:
Unexpected key(s) in state_dict: “1.4.bias”, “1.8.bias”.

Can someone give me a tip on how to solve this issue?

Thanks a lot in advance!

Christoph

2

Are you sure they are the exact same? And the model was not updated? Because your keys are different. This is not specifically a fastai2 issue as it’s saying there are layers in the state_dict that the model was not expecting.

3

Thanks for the fast reply!
You were right. I found the difference.
It was in the custom head, that fastai adds to the model.

In fastai1 the following head is added:

(1): Sequential(
    (0): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (1): Flatten()
    (2): BatchNorm1d(4096, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=4096, out_features=512, bias=False)
    (5): ReLU(inplace=True)
    (6): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (7): Dropout(p=0.5, inplace=False)
    (8): Linear(in_features=512, out_features=4, bias=False)
  )

In fastai2:

(1): Sequential(
    (0): AdaptiveConcatPool2d(
      (ap): AdaptiveAvgPool2d(output_size=1)
      (mp): AdaptiveMaxPool2d(output_size=1)
    )
    (1): Flatten(full=False)
    (2): BatchNorm1d(4096, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (3): Dropout(p=0.25, inplace=False)
    (4): Linear(in_features=4096, out_features=512, bias=True)
    (5): ReLU(inplace=True)
    (6): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (7): Dropout(p=0.5, inplace=False)
    (8): Linear(in_features=512, out_features=4, bias=True)
  )

The difference is in layer 4 and 8 of the the head. Knowing that, the error message now makes a lot more sense too.
In fastai1 the boolean value “bias” is set to True, in fastai2 it is set to False.
I simply replaced the two layers and now it is loading the weights correctly.

1 Like