Learn.summary() RunTimeError in Language Model

Hi
I’m trying to run my own versions of the examples on this notebook but I cant get Learn.summary() to work. It keeps giving me an error saying my hidden size is wrong:

RuntimeError                              Traceback (most recent call last)

<ipython-input-99-bc39e9e85f86> in <module>()
----> 1 learn.summary()

8 frames

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py in check_hidden_size(self, hx, expected_hidden_size, msg)
    174         # type: (Tensor, Tuple[int, int, int], str) -> None
    175         if hx.size() != expected_hidden_size:
--> 176             raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size())))
    177 
    178     def check_forward_args(self, input, hidden, batch_sizes):

RuntimeError: Expected hidden size (1, 1, 200), got (1, 64, 200)

If I understood correctly, (1,1,200) means 1 GRU layer, batch size of 1 and embedding size of 200 However, even if I hardcode these values learn.summary() doesn’t work. In fact, it gives me another error:

AttributeError: 'list' object has no attribute 'shape'

What is odd is that I can actually train the model perfectly fine. I also checked Pytorch forums and other people had the same problem. Anybody got an idea of how to solve this?

Here’s my arch:

nv = 55
nh = 200
bs = 64
    class Model5(nn.Module):
        def __init__(self):
            super().__init__()
            self.i_h = nn.Embedding(nv,nh)
            self.rnn = nn.GRU(nh, nh, 1, dropout=0.0, batch_first=True)
            self.h_o = nn.Linear(nh,nv)
            self.bn = BatchNorm1dFlat(nh)
            self.h = torch.zeros(1, bs, nh).cuda()
            
        def forward(self, x):
            res,h = self.rnn(self.i_h(x), self.h)
            self.h = h.detach()
            return self.h_o(self.bn(res))

Just one more thing: how can I save the encoder of a custom language model?

Can you provide the complete full stack trace (not just 1 out of the 8 frames)?

Perhaps take a look at the lesson3-imdb notebook which uses learn.save_encoder().

Hi Phillip
This is the complete error when I call learn.summary():

---------------------------------------------------------------------------

RuntimeError                              Traceback (most recent call last)

<ipython-input-112-d408ef516abc> in <module>()
  1 learn.model;
----> 2 learn.summary()


/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py in check_hidden_size(self, hx, expected_hidden_size, msg)
174         # type: (Tensor, Tuple[int, int, int], str) -> None
175         if hx.size() != expected_hidden_size:
--> 176             raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size())))
177 
178     def check_forward_args(self, input, hidden, batch_sizes):

RuntimeError: Expected hidden size (1, 1, 200), got (1, 64, 200)

If I hardcode hidden size as the error says ((1, 1, 200)), it gives me this other error:

---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)

<ipython-input-115-d408ef516abc> in <module>()
      1 learn.model;
----> 2 learn.summary()

/usr/local/lib/python3.6/dist-packages/fastai/callbacks/hooks.py in <listcomp>(.0)
    148         with hook_params(flatten_model(m))as hook_p:
    149             x = m.eval()(*x) if is_listy(x) else m.eval()(x)
--> 150             output_size = [((o.stored.shape[1:]) if o.stored is not None else None) for o in hook_o]
    151             params = [(o.stored if o.stored is not None else (None,None)) for o in hook_p]
    152     params, trainables = map(list,zip(*params))

AttributeError: 'list' object has no attribute 'shape'

The same happens if I run the example GRU’s on lesson7 notebooks.