When I use pre-defined model of fastai like resnet34, … I can run multiple learning rate training.
But when I self-define a deep network, it is just able to run normal training(constant learning rate or SGD with restarts), not able to run multiple learning rate.
It returns error:
According to my study, the pre-defined networks are constructed by 3 blocks of layers, so they can be trained by 3 individual learning rates for 3 individual blocks.
So I want to ask, how to code a network into blocks to run multiple learning rate training?
I’m afraid I don’t have a specific answer for you, but I’ve noticed something odd. I tried searching the fastai github for the code referenced in the screenshot you posted. The learner.py file it mentions is in a directory called ‘old’, and it looks to me like it’s not used anymore. Is it possible you’re running an outdated version of the library? You might have better luck (or get better answers) if you try updating the library. This may break other parts of your code though if the API has changed significantly since then.
Thank @JoelH. Honestly I can not run the new library of fastai, so I used the old version.
It took me a lot of time to debug and study the fastai library, now I know how to train multi-lr now.
Firstly, we have to call len(learn.models.get_layer_groups()) to obtain the number of groups in our network: N
Afterward, based on the number of groups, we create the learning rates array with length of N
