LanguageModelLoader - 'numpy.int64' object has no attribute 'text'

Part 2 & Alumni (2018)
#1

hi,
I am new to NLP. And, I am trying to replicate the results in fastai/courses/dl2/imdb.ipynb.

when I try to execute

trn_dl = LanguageModelLoader(np.concatenate(trn_lm), bs, bptt)

I get below error:

AttributeError: ‘numpy.int64’ object has no attribute ‘text’

It seems a valid error since trn_lm is having only numbers.

trn_lm = np.array([[stoi[o] for o in p] for p in tok_trn])

Has anyone come across this error before? Please let me know how to resolve this

i looked at LanguageModelLoader class definition.I am not sure where ‘o.text’ is coming from.

Class DataLanugageLoader():

def __init__(self, ds, bs, bptt, backwards=False):
    self.bs,self.bptt,self.backwards = bs,bptt,backwards
    text = sum([o.text for o in ds], [])
    fld = ds.fields['text']
    nums = fld.numericalize([text],device=None if torch.cuda.is_available() else -1)
    self.data = self.batchify(nums)
    self.i,self.iter = 0,0
    self.n = len(self.data)

P.S. I did not install fastai v 0.7. Instead I am using only those functions which are required to execute the code in IMDB notebook.

#2

I am trying to install fastai 0.7

pip install fastai==0.7.0
pip install torchtext==0.2.3
from fastai.text import *

I get below error:

----> 1 from fastai.text import *
2 import html

~/notebooks/notebooks/fastai/courses/dl2/fastai/text.py in
1 from .core import *
----> 2 from .learner import *
3 from .lm_rnn import *
4 from torch.utils.data.sampler import Sampler
5 import spacy

~/notebooks/notebooks/fastai/courses/dl2/fastai/learner.py in
4 from .transforms import *
5 from .model import *
----> 6 from .dataset import *
7 from .sgdr import *
8 from .layer_optimizer import *

~/notebooks/notebooks/fastai/courses/dl2/fastai/dataset.py in
185 return full_names, label_arr, all_labels
186
–> 187 class BaseDataset(Dataset):
188 “”“An abstract class representing a fastai dataset. Extends torch.utils.data.Dataset.”""
189 def init(self, transform=None):

NameError: name ‘Dataset’ is not defined

#3

I got this resolved. Updated code is in Lesson 10 video.