I’m instantiating my learner like so:
learner = tabular_learner(data, layers=[1000,500], emb_drop=0.2, ps=[0.2, 0.2], metrics=[metrics.accuracy, metrics.AUROC()])
When I do learner.summary() I get this:
< A bunch of embeddings> ____________________________________________________________________
Dropout [2157] 0 False
BatchNorm1d [383] 766 True
Linear [1000] 2,541,000 True
ReLU [1000] 0 False
BatchNorm1d [1000] 2,000 True
Dropout [1000] 0 False
Linear [500] 500,500 True
ReLU [500] 0 False
BatchNorm1d [500] 1,000 True
Dropout [500] 0 False
Linear [2] 1,002 True
One thing that jumped out at me is that the first BatchNorm1d layer has a really small output size. Why is that? It jumps back up to 1000, which is what I’d expect, in the next layer.
I expected the BatchNorm to have the same output size as the previous dropout layer, which is 2157, not 383.
Any insight would be appreciated.