Loaded state dict contains a parameter group that doesn't match the size of optimizer's group ... after unfreeze and then lr_find

Part 1 (2019)
1

Running my language model on 1.0.46 and getting the following exception after unfreezing and running lr_find on my language_model_learner:

learn.unfreeze()
learn.lr_find(start_lr=1e-3/10, end_lr=1e-2*10, wd=wd)
learn.recorder.plot()

… results in the following exception:

ValueError: loaded state dict contains a parameter group that doesn’t match the size of optimizer’s group

1 Like
2

Having the same problem

3

I’m having the same problem (but not with a language model, just training a text classifier).

4

I’m pretty sure I also had this one, then I ran again and it disappeared

1 Like
5

Yah, this is the behavior I’ve noticed as well.

6

Having this problem using a text classifier, but the error doesn’t come consistently even with rerunning.

7

I only had the problem if I used lr_find at any point after using predict.

So the following returned an error:

learn = language_model_learner(data, model, drop_mult=0.3)
print(learn.predict('I went to buy some food', 100))
learn.lr_find()

However, if I just reload (or recreate) the language_model_learner between predict and lr_find then it works:

learn = language_model_learner(data, model, drop_mult=0.3)
print(learn.predict('I went to buy some food', 100))
learn = language_model_learner(data, model, drop_mult=0.3)
learn.lr_find()

Admittedly quite hacky, but a quick solution.