Label_from_df: type promotion error?

Currently trying to create a tabular model with all categorical variables predicting a boolean. My data is in an identical format to Rossman going in, but having an issue with the label_from_df method.

Here’s the error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-d1d1aaded1de> in <module>
      1 data = (TabularList.from_df(df, path=PATH, cat_names=cat_names, procs=procs)
      2                            .random_split_by_pct()
----> 3                            .label_from_df(cols=dep_var)
      4                            .add_test(test)
      5                            .databunch())

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in _inner(*args, **kwargs)
    432             self.valid = fv(*args, **kwargs)
    433             self.__class__ = LabelLists
--> 434             self.process()
    435             return self
    436         return _inner

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in process(self)
    482         "Process the inner datasets."
    483         xp,yp = self.get_processors()
--> 484         for ds,n in zip(self.lists, ['train','valid','test']): ds.process(xp, yp, name=n)
    485         #progress_bar clear the outputs so in some case warnings issued during processing disappear.
    486         for ds in self.lists:

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in process(self, xp, yp, name)
    656                     p.warns = []
    657                 self.x,self.y = self.x[~filt],self.y[~filt]
--> 658         self.x.process(xp)
    659         return self
    660 

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in process(self, processor)
     71         if processor is not None: self.processor = processor
     72         self.processor = listify(self.processor)
---> 73         for p in self.processor: p.process(self)
     74         return self
     75 

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/tabular/data.py in process(self, ds)
     72             ds.codes = np.stack([c.cat.codes.values for n,c in ds.xtra[ds.cat_names].items()], 1).astype(np.int64) + 1
     73             self.classes = ds.classes = OrderedDict({n:np.concatenate([['#na#'],c.cat.categories.values])
---> 74                                       for n,c in ds.xtra[ds.cat_names].items()})
     75             cat_cols = list(ds.xtra[ds.cat_names].columns.values)
     76         else: ds.codes,ds.classes,self.classes,cat_cols = None,None,None,[]

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/tabular/data.py in <dictcomp>(.0)
     72             ds.codes = np.stack([c.cat.codes.values for n,c in ds.xtra[ds.cat_names].items()], 1).astype(np.int64) + 1
     73             self.classes = ds.classes = OrderedDict({n:np.concatenate([['#na#'],c.cat.categories.values])
---> 74                                       for n,c in ds.xtra[ds.cat_names].items()})
     75             cat_cols = list(ds.xtra[ds.cat_names].columns.values)
     76         else: ds.codes,ds.classes,self.classes,cat_cols = None,None,None,[]

TypeError: invalid type promotion

Would love any insight into what’s going on in the source code here:

 if len(ds.cat_names) != 0:
            ds.codes = np.stack([c.cat.codes.values for n,c in ds.inner_df[ds.cat_names].items()], 1).astype(np.int64) + 1
            self.classes = ds.classes = OrderedDict({n:np.concatenate([['#na#'],c.cat.categories.values])
                                      for n,c in ds.inner_df[ds.cat_names].items()})
            cat_cols = list(ds.inner_df[ds.cat_names].columns.values)

Figured out the issue. I had dates with dtype datetime64[ns] being passed in as a categorical variable. Used the add_datepart method and dropped the original columns. Works like a charm!

3 Likes

Hello,
I’m a beginner to coding.
I am getting the same kind of error, an i understand that it is because of date,but i don’t know what to do, could you elaborate on what you did?