Hi
I’m trying to run my own versions of the examples on this notebook but I cant get Learn.summary() to work. It keeps giving me an error saying my hidden size is wrong:
RuntimeError Traceback (most recent call last)
<ipython-input-99-bc39e9e85f86> in <module>()
----> 1 learn.summary()
8 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/rnn.py in check_hidden_size(self, hx, expected_hidden_size, msg)
174 # type: (Tensor, Tuple[int, int, int], str) -> None
175 if hx.size() != expected_hidden_size:
--> 176 raise RuntimeError(msg.format(expected_hidden_size, tuple(hx.size())))
177
178 def check_forward_args(self, input, hidden, batch_sizes):
RuntimeError: Expected hidden size (1, 1, 200), got (1, 64, 200)
If I understood correctly, (1,1,200) means 1 GRU layer, batch size of 1 and embedding size of 200 However, even if I hardcode these values learn.summary() doesn’t work. In fact, it gives me another error:
AttributeError: 'list' object has no attribute 'shape'
What is odd is that I can actually train the model perfectly fine. I also checked Pytorch forums and other people had the same problem. Anybody got an idea of how to solve this?
Here’s my arch:
nv = 55
nh = 200
bs = 64
class Model5(nn.Module):
def __init__(self):
super().__init__()
self.i_h = nn.Embedding(nv,nh)
self.rnn = nn.GRU(nh, nh, 1, dropout=0.0, batch_first=True)
self.h_o = nn.Linear(nh,nv)
self.bn = BatchNorm1dFlat(nh)
self.h = torch.zeros(1, bs, nh).cuda()
def forward(self, x):
res,h = self.rnn(self.i_h(x), self.h)
self.h = h.detach()
return self.h_o(self.bn(res))
Just one more thing: how can I save the encoder of a custom language model?