Someone should double check this, but I think fastai2 applies it all the weights and biases. I was digging through a while back, and saw all the model parameters(weights+biases) got updated with the decay. I saw “for p in model.parameters()”. etc. etc.
Conventionally, weight decay (l2 regularization) is applied only to the weights. You can add it to the bias term as well, but just applying on the weights seems good enough for regularization.
Source: Andrew Ng regularization lecture