Learn.summary() suggests ReLu output is the wrong size?

Hello!

I have been playing around with Collaborative Filtering a bit with the use_nn=True flag set. I wanted to get a bit more information about the size of the embeddings and the layers etc so I used Learn.summary() (I’ve truncated the output slightly):

learn = collab_learner(dls, use_nn=True, y_range=(0, 5.5), layers=[100,50])
learn.summary()
================================================================
Layer (type)         Output Shape         Param #    Trainable 
================================================================
Embedding            32768 x 363          5,853,738  True      
________________________________________________________________
Embedding            32768 x 430          9,356,370  True      
________________________________________________________________
Dropout              32768 x 793          0          False     
________________________________________________________________
BatchNorm1d          32768 x 793          1,586      True      
________________________________________________________________
Linear               32768 x 100          79,300     True      
________________________________________________________________
ReLU                 32768 x 50           0          False     
________________________________________________________________
BatchNorm1d          32768 x 100          200        True      
________________________________________________________________
Linear               32768 x 50           5,000      True      
________________________________________________________________
ReLU                 32768 x 50           0          False     
________________________________________________________________
Linear               32768 x 1            51         True      
________________________________________________________________
SigmoidRange         32768 x 1            0          False     
________________________________________________________________

This suggests that the ReLu after the first linear layer has an output length of 50, however the first layer has an output size of 100, as does the following BatchNorm layer.

Am I missing something here or should the first ReLu output not have a size of 32768 x 100?

Many thanks!

1 Like