Hi @devforfu, Thanks for double checking! Perhaps you can merge your layer group 2 and 3? If people use lr_range to get the learning rates, then the layer group 2 will have much smaller lr than group 3, so forth for the custom head layers. It doesn’t quite make sense why the last “Fire” layer is so special in this split?..
Probably it doesn’t matter that much since people can set learning rates to be whatever they want