Fastbook Chapter 12 questionnaire (wiki)

Part 1 (2020)
2

Question 5:

We should change the model to remove the third word from training in forward method if our sequence length is 2, no?

Like this:

class LMModel1_Modified(Module):
    def __init__(self, vocab_sz, n_hidden):
        self.i_h = nn.Embedding(vocab_sz, n_hidden)  
        self.h_h = nn.Linear(n_hidden, n_hidden)     
        self.h_o = nn.Linear(n_hidden,vocab_sz)
        
    def forward(self, x):
        h = F.relu(self.h_h(self.i_h(x[:,0])))
        h = h + self.i_h(x[:,1])
        h = F.relu(self.h_h(h))
        return self.h_o(h)

Also, as per the question, sequence should also be changed:

seqs = L((tensor(nums[i:i+2]), nums[i+2]) for i in range(0,len(nums)-4, 2))

is this right? or am I missing something?

2 Likes