Fastai v2 default layer initialization

I was trying to find if fastai v2 does any layer initialization for us by default. For example with nn.init.kaiming_normal. From my quick code search it doesn’t appear so, but I wanted to double check.

Thanks,

It does initialize convolution layers in ConvLayer and fastai’s Embedding also has a custom init.

1 Like

Should layers in a tabular model or any Linear layer be initialized manually then as a best practice? For example with kaiming normal?

Thanks!

If you want to.

I thought PyTorch might be doing reasonable init nowadays by default - worth double-checking that.

2 Likes

Looking at the code in pytorch, seems like it is using kaiming uniform:

1 Like