How does the PoolingLinearClassifier get its name? It’s not a linear combination of features and weights. Sure it has l Linear Layers, but w/ a non-linear Relu in-between.
https://docs.fast.ai/text.models.html#PoolingLinearClassifier
Pooling because it concatenates the last hidden state, plus a maxpool and avgpool over all hidden states.
It also does have nonlinearities. Batchnorm layers as well.
Karl,
Thanks for your response. It’s very clear from the code the model has Pooling. However, what is not clear to me to is why it has ‘Linear’ in the name. Why?
If you do a quick look at the source code, you would realize that the output of concatenation is passed through a linear layer bn_drop_lin.
it is passed through a sequence of bin_drop_lin layers which are composed of (bn, dropout, matrix_mul, ReLU) which is not a linear operation.
Wouldn’t a linear classifier draw decision boundaries based on a linear combination of features and weights? This isn’t that.