MLP using fastai and IMDB dataset

Part 1 (2019)
1

Hello,
I’m trying to create a simple Multilayer Perceptron using fastai library to classify IMDB reviews.

Here is the code I’m using. However, at the moment I’m stuck with the size of the input output to each linear layer. I’m not sure I really understand why. Grateful if someone could give me some hints.

from fastai.text import *
path = untar_data(URLs.IMDB_SAMPLE)
path.ls()

data = TextClasDataBunch.from_csv(path, 'texts.csv')

x,y = next(iter(data.train_dl))
x.shape, y.shape

# Multilayer perceptron
class MultilayerPerceptron(nn.Module):
def __init__(self):
    super().__init__()
    self.lin1 = nn.Linear(1503, 250, bias=True) 
    self.lin2 = nn.Linear(250, 120, bias=True)
    self.lin3 = nn.Linear(120, 2, bias=True)

def forward(self, xb):
    x = xb.float()
    #x = xb.view(250, -1)
    x = F.relu(self.lin1(x))
    x = F.relu(self.lin2(x))
    return self.lin3(x)

mlp_learner = Learner(data=data, model=MultilayerPerceptron(), loss_func=nn.CrossEntropyLoss(),metrics=accuracy)
mlp_learner.fit_one_cycle(5,1e-2)

However, the code return the error: RuntimeError: size mismatch, m1: [64 x 178], m2: [1503 x 250] at /opt/conda/conda-bld/pytorch_1565272271120/work/aten/src/THC/generic/THCTensorMathBlas.cu:273

It seems to me that it has something to do with input and out of hidden layers but not clear why. The application of MLP should be quiet straightforward.

Thank you.

2

I think the issue is that I need an embedding layer at the beginning of the MLP.
However, I cannot find resources that can help me with that. Has anyone got any advice on how to proceed? Thank you.

3

Use torch.nn.Embedding

4

Thanks @kushaj.
I came up with the following:

class Model1(nn.Module):
    def __init__(self):
        super().__init__()
        self.i_h = nn.Embedding(nv,nh) 
        self.h_h = nn.Linear(nh,nh, bias = True)    
        self.h_o = nn.Linear(nh,nv, bias = True)    
            
    def forward(self, x):
        h = torch.zeros(x.shape[0], nh).to(device=x.device)
        for i in range(x.shape[1]):
            h = h + self.i_h(x[:, i])
            h = F.relu(self.h_h(h))
        return self.h_o(h)

This should do the trick for classification of review sentiment. Agree? any feedback?
Thank you again for your answer.

1 Like