Classification with a lot of the classes

Part 1 (2019)
1

Hello,

I’m trying to make model which will classify text into about 500 different classes. I think that I have to customize architecture of the Pooling Classifier which looks now like this:

(1): PoolingLinearClassifier(
(layers): Sequential(
  (0): BatchNorm1d(1200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (1): Dropout(p=0.2, inplace=False)
  (2): Linear(in_features=1200, out_features=50, bias=True)
  (3): ReLU(inplace=True)
  (4): BatchNorm1d(50, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): Dropout(p=0.1, inplace=False)
  (6): Linear(in_features=50, out_features=498, bias=True)
)

I think that I have to change in (2): Linear layer to have more out_features because in the last (6) Linear layer I predict more out_features than I’ve got in_features. What do you think?

Best regards

2

Is it wrong that I want to predict 498 classes from only 50 “in_features”, isn’t it?

3

Yes.
I suggest this.
Sequential(
(0): BatchNorm1d(1200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(1): Dropout(p=0.2, inplace=False)
(2): Linear(in_features=1200, out_features=800, bias=True)
(3): ReLU(inplace=True)
(4): BatchNorm1d(800, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Dropout(p=0.1, inplace=False)
(6): Linear(in_features=800, out_features=498, bias=True)
)

4

@JonathanSum, thank you for answer. Do you know how can I change PoolingClassifier when I use fastai?

Best regards