I’m trying to run my own versions of the examples on this notebook but I cant get Learn.summary() to work. It keeps giving me an error saying my hidden size is wrong:
RuntimeError Traceback (most recent call last) <ipython-input-99-bc39e9e85f86> in <module>() ----> 1 learn.summary() 8 frames /usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py in check_hidden_size(self, hx, expected_hidden_size, msg) 174 # type: (Tensor, Tuple[int, int, int], str) -> None 175 if hx.size() != expected_hidden_size: --> 176 raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size()))) 177 178 def check_forward_args(self, input, hidden, batch_sizes): RuntimeError: Expected hidden size (1, 1, 200), got (1, 64, 200)
If I understood correctly, (1,1,200) means 1 GRU layer, batch size of 1 and embedding size of 200 However, even if I hardcode these values learn.summary() doesn’t work. In fact, it gives me another error:
AttributeError: 'list' object has no attribute 'shape'
What is odd is that I can actually train the model perfectly fine. I also checked Pytorch forums and other people had the same problem. Anybody got an idea of how to solve this?
Here’s my arch:
nv = 55 nh = 200 bs = 64 class Model5(nn.Module): def __init__(self): super().__init__() self.i_h = nn.Embedding(nv,nh) self.rnn = nn.GRU(nh, nh, 1, dropout=0.0, batch_first=True) self.h_o = nn.Linear(nh,nv) self.bn = BatchNorm1dFlat(nh) self.h = torch.zeros(1, bs, nh).cuda() def forward(self, x): res,h = self.rnn(self.i_h(x), self.h) self.h = h.detach() return self.h_o(self.bn(res))
Just one more thing: how can I save the encoder of a custom language model?