Create a learner with no validation set

Hey! I am trying to create a Learner object with no validation set but I am getting a weird problem with the classes attribute:

TypeError                                 Traceback (most recent call last)
/opt/anaconda3/lib/python3.6/site-packages/IPython/core/formatters.py in __call__(self, obj)
    700                 type_pprinters=self.type_printers,
    701                 deferred_pprinters=self.deferred_printers)
--> 702             printer.pretty(obj)
    703             printer.flush()
    704             return stream.getvalue()

/opt/anaconda3/lib/python3.6/site-packages/IPython/lib/pretty.py in pretty(self, obj)
    400                         if cls is not object \
    401                                 and callable(cls.__dict__.get('__repr__')):
--> 402                             return _repr_pprint(obj, self, cycle)
    403 
    404             return _default_pprint(obj, self, cycle)

/opt/anaconda3/lib/python3.6/site-packages/IPython/lib/pretty.py in _repr_pprint(obj, p, cycle)
    695     """A pprint that just redirects to the normal repr function."""
    696     # Find newlines and replace them with p.break_()
--> 697     output = repr(obj)
    698     for idx,output_line in enumerate(output.splitlines()):
    699         if idx:

/opt/anaconda3/lib/python3.6/site-packages/dataclasses.py in __repr__(self)

~/fastai/fastai/basic_data.py in __repr__(self)
     96 
     97     def __repr__(self)->str:
---> 98         return f'{self.__class__.__name__};\nTrain: {self.train_ds};\nValid: {self.valid_ds};\nTest: {self.test_ds}'
     99 
    100     @classmethod

~/fastai/fastai/data_block.py in __repr__(self)
    455     def __repr__(self)->str:
    456         x = f'{self.x}' # force this to happen first
--> 457         return f'{self.__class__.__name__}\ny: {self.y}\nx: {x}'
    458     def predict(self, res):
    459         "Delegates predict call on `res` to `self.y`."

~/fastai/fastai/data_block.py in __repr__(self)
     56         return self.items[i]
     57     def __repr__(self)->str:
---> 58         items = [self[i] for i in range(min(5,len(self.items)))]
     59         return f'{self.__class__.__name__} ({len(self)} items)\n{items}...\nPath: {self.path}'
     60 

~/fastai/fastai/data_block.py in <listcomp>(.0)
     56         return self.items[i]
     57     def __repr__(self)->str:
---> 58         items = [self[i] for i in range(min(5,len(self.items)))]
     59         return f'{self.__class__.__name__} ({len(self)} items)\n{items}...\nPath: {self.path}'
     60 

~/fastai/fastai/data_block.py in __getitem__(self, idxs)
     88 
     89     def __getitem__(self,idxs:int)->Any:
---> 90         if isinstance(try_int(idxs), int): return self.get(idxs)
     91         else: return self.new(self.items[idxs], xtra=index_row(self.xtra, idxs))
     92 

~/fastai/fastai/data_block.py in get(self, i)
    294         o = self.items[i]
    295         if o is None: return None
--> 296         return Category(o, self.classes[o])
    297 
    298     def analyze_pred(self, pred, thresh:float=0.5): return pred.argmax()

TypeError: 'NoneType' object is not subscriptable

This means that it is not finding the classes attribute. I built my ImageDataBunch like this:

np.random.seed(42)
src_rel = (ImageItemList.from_folder(path) #Where to find the data? -> in path and its subfolders
           .label_from_folder())           #How to label? -> depending on the folder of the filenames

data_rel = ImageDataBunch.create(src_rel, valid_ds=src_rel, path=path)

And tried many ways to set my classes:

data_rel.classes = data.classes
data_rel.train_ds.classes = data.classes
data_rel.valid_ds.classes = data.classes

And even this before creating the ImageDataBunch:

src_rel.classes = data.classes

but to no avail, still getting the same error. Any help is appreciated!

1 Like

Hey,I am getting the same error.
How did you solve it?

You should use no_split() in the data block API.

3 Likes

Hey,I tried using no_splits() but still get the same error at the time of inference

TypeError: ‘NoneType’ object is not subscriptable

Hey,I figured it out.The line given below is the reason for the error.

learner = load_learner(PATH, fname = ‘export_clas.pkl’)

if we remove this line from the inference step it works fine.

So, at inference we only have to do the following

bs=20
data_class = TextClasDataBunch.load(PATH, ‘tmp_clas’, bs=bs)
learner = text_classifier_learner(data_class).load(‘model-name’, with_opt=False)
learner.predict(“Hey,this was a great movie”)

Ah good catch! Learner.export() uses the validation set to export the state (for transforms for instance) so it should default on the training set when there is none.

Can you give your exact code? I can’t reproduce your bug. Even with no_split, Learner.export() works well for me.

yes but learner.predict() gives an error

Again, I don’t have that bug, learner.predict() works fine for me.