Lesson 8 - Official topic

Any solution for this issue?

I’m unable to save weights. Any thoughts on what to do?

Hi Megan! Would love to join! Will there be a meet today or tomorrow?

Sounds good, Dan. I’m taking this week off course review to work on a covid hackathon. Meetings start next week.

1 Like

Can it be @harish3110 that in this example you mentioned your data had only 3 columns: text, sentiment and is_valid?

It seems to me know this could be the key but I am not entirely sure.

sentencepiece throws an error (see below) when I run notebook 10_nlp.ipynb in Google Colab. Has anyone managed to successfully deal with this issue?

1 Like

In Chapter 12 we create an LSTMCell from scratch as such:

class LSTMCell(Module):
    def __init__(self, ni, nh):
        self.ih = nn.Linear(ni,4*nh)
        self.hh = nn.Linear(nh,4*nh)

    def forward(self, input, state):
        h,c = state
        #One big multiplication for all the gates is better than 4 smaller ones
        gates = (self.ih(input) + self.hh(h)).chunk(4, 1)
        ingate,forgetgate,outgate = map(torch.sigmoid, gates[:3])
        cellgate = gates[3].tanh()

        c = (forgetgate*c) + (ingate*cellgate)
        h = outgate * c.tanh()
        return h, (h,c)

How do I create a single layer LSTM model to use this cell? This isn’t implemented in the notebook. I have tried it on my own but I’m getting an error RuntimeError: size mismatch, m1: [1024 x 64], m2: [2 x 256] at /pytorch/aten/src/TH/generic/THTensorMath.cpp:41 and I’m not sure what the mistake is. Here’s the code for the model and training:

class LMModel6(Module):
    def __init__(self, vocab_sz, n_hidden, n_layers):
        self.i_h = nn.Embedding(vocab_sz, n_hidden)
        self.lstm = LSTMCell(n_layers, n_hidden)
        self.h_o = nn.Linear(n_hidden, vocab_sz)
        self.h = torch.zeros(n_layers, bs, n_hidden)
    def forward(self, x):
        h, res = self.lstm(self.i_h(x), self.h)
        self.h = h.detach()
        return self.h_o(res)
    def reset(self):
      for h in self.h: h.zero_()
learn = Learner(dls, LMModel6(len(vocab), 64, 2), 
                metrics=accuracy, cbs=ModelReseter)
learn.fit_one_cycle(15, 1e-2)

Any idea how to fix this?

Can you share output of learn.summary?

Learn.summary won’t work since there is a runtime error.

But you should be able to see the inputs and outputs and then dwelve into the code to check. This is just a suggestion.

Good morning catanza hope you are having a wonderful day.

I am receiving the exact error!

I use the following to repositories at the start of my notebook as this is what, has worked for me.


from colab_utils import setup_fastai_colab setup_fastai_colab()

I had to install !pip install sentencepiece this morning and have had the error since then, it is my first attempt at NLP so am a little stuck :confused:.

Have you managed to find a resolution for this error?

Kind regards mrfabulous1 :grinning: :grinning:

HI jcatanza hope your having a Fun day!

I was trying to resolve this problem on another thread!

The above solution worked for me if you still have the issue!

Cheers mrfabulous! :grinning: :grinning:

1 Like

Thanks @mrfabulous1 I installed an earlier version of sentencepiece
% pip install sentencepiece==0.1.86, and now there is no problem.


@Salazar we’ll start next week May 26th Tuesday, 6-9pm PST. We will silently work on notebooks, then discuss as a group. You don’t have to complete anything in advance. URL to join next week’s meeting: https://meet.google.com/hgw-itjd-hep. Hope to see some of you there!


I am using paper space and I get the following error when running this line of code in notebook 10. I am not sure why it is looking for a pkl file.

dls_lm = DataBlock(
blocks=TextBlock.from_folder(path, is_lm=True),
get_items=get_imdb, splitter=RandomSplitter(0.1)
).dataloaders(path, path=path, bs=128, seq_len=80)

It’s due to the tokenized text information being saved away. This may be a similar issue to what has been going on with the old course and paperspace, where the system path would give issues. A better place for this may be on the paperspace platform discussion? (Where I know mods from paperspace are on often):

Thanks. I will post there.

Is this working for you? I haven’t been able to get it to work in fastai2

How are you passing it in? On Learner or on your call to fit?

Does Fastai2 support multi-gpu training?