TypeError when using Simple Tabular model (get_tabular_learner)

fastai fastai dev
1 
=== Software ===
python version : 3.7.0
fastai version : 1.0.22
torch version  : 1.0.0.dev20181112
torch cuda ver
torch cuda is  : **Not available**

=== Hardware ===
No GPUs available

=== Environment ===
platform       : Linux-4.15.0-34-generic-x86_64-with-debian-buster-sid
distro         : #37-Ubuntu SMP Mon Aug 27 15:21:48 UTC 2018
conda env      : fastai
python         : ~/miniconda3/envs/fastai/bin/python
sys.path       :
~/miniconda3/envs/fastai/lib/python37.zip
~/miniconda3/envs/fastai/lib/python3.7
~/miniconda3/envs/fastai/lib/python3.7/lib-dynload
~/miniconda3/envs/fastai/lib/python3.7/site-packages
~/miniconda3/envs/fastai/lib/python3.7/site-packages/IPython/extensions
no supported gpus found on this system

My program is a simple test of the tabular methods:

from fastai import *
from fastai.tabular import *
n=np.random.random((10,5))
df=pd.DataFrame(n,columns=[‘val’,‘o1’,‘o2’,‘o3’,‘o4’])
train_df = df.copy()[:7]
valid_df = df.copy()[-3:]
pth=Path(’.’)
data = TabularDataBunch.from_df(pth, train_df, valid_df, “val”)

The error is from running the following command:
learn = get_tabular_learner(data, layers=[5,5],metrics=accuracy,use_bn=False)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-33-b81eaa87b01c> in <module>
----> 1 learn = get_tabular_learner(data, layers=[5,5],metrics=accuracy,tfms=[FillMissing],use_bn=False)

~/miniconda3/envs/fastai/lib/python3.7/site-packages/fastai/tabular/data.py in get_tabular_learner(data, layers, emb_szs, metrics, ps, emb_drop, y_range, use_bn, **kwargs)
     95     emb_szs = data.get_emb_szs(ifnone(emb_szs, {}))
     96     model = TabularModel(emb_szs, len(data.cont_names), out_sz=data.c, layers=layers, ps=ps, emb_drop=emb_drop,
---> 97                          y_range=y_range, use_bn=use_bn)
     98     return Learner(data, model, metrics=metrics, **kwargs)
     99 

~/miniconda3/envs/fastai/lib/python3.7/site-packages/fastai/tabular/models.py in __init__(self, emb_szs, n_cont, out_sz, layers, ps, emb_drop, y_range, use_bn)
     20         layers = []
     21         for i,(n_in,n_out,dp,act) in enumerate(zip(sizes[:-1],sizes[1:],[0.]+ps,actns)):
---> 22             layers += bn_drop_lin(n_in, n_out, bn=use_bn and i!=0, p=dp, actn=act)
     23         self.layers = nn.Sequential(*layers)
     24 

~/miniconda3/envs/fastai/lib/python3.7/site-packages/fastai/layers.py in bn_drop_lin(n_in, n_out, bn, p, actn)
     31     layers = [nn.BatchNorm1d(n_in)] if bn else []
     32     if p != 0: layers.append(nn.Dropout(p))
---> 33     layers.append(nn.Linear(n_in, n_out))
     34     if actn is not None: layers.append(actn)
     35     return layers

~/miniconda3/envs/fastai/lib/python3.7/site-packages/torch/nn/modules/linear.py in __init__(self, in_features, out_features, bias)
     46         self.in_features = in_features
     47         self.out_features = out_features
---> 48         self.weight = Parameter(torch.Tensor(out_features, in_features))
     49         if bias:
     50             self.bias = Parameter(torch.Tensor(out_features))

TypeError: new() received an invalid combination of arguments - got (NoneType, int), but expected one of:
 * (torch.device device)
 * (torch.Storage storage)
 * (Tensor other)
 * (tuple of ints size, torch.device device)
      didn't match because some of the arguments have invalid types: (NoneType, int)
 * (object data, torch.device device)
      didn't match because some of the arguments have invalid types: (NoneType, int)

What is the reason for this error? The data is all float64. I also tried changing all the data to float32. Not sure where the (NoneType, int) is coming from. Btw, The tabular examples from fastai/examples works fine.

Any help is greatly appreciated.

Thank you!

2

I think I see what the issue is – it looks like when I change my dependent variable to be of type integer, and perform classification, the tabular methods work fine. But they still do not work when the dependent variable is of type float64.

Looks like I’m missing something when trying to run a tabular regression. How do I run a tabular regression?

Thank you!

3

Hi yeehaw2 – could you share how you cast your dependent variable please? I’m having the exact same issue but casting variables as float32 or int32 didn’t fix it.

Thanks

1 Like
4

Here’s what i’ve been trying:


from fastai import *
from fastai.tabular import * 
from fastai import *
from fastai.tabular import * 

; n
n=np.random.random((5,5)); n
array([[0.111232, 0.337403, 0.659756, 0.652794, 0.456497],
       [0.124652, 0.319763, 0.463694, 0.956675, 0.189138],
       [0.393637, 0.975163, 0.016941, 0.066641, 0.932285],
       [0.528168, 0.388681, 0.878955, 0.310274, 0.649157],
       [0.63696 , 0.851295, 0.954441, 0.762749, 0.822939]])

df=pd.DataFrame(n,columns=['val','o1','o2','o3','o4'])

#
#df["val"]=(df["o1"]>0.5).astype("int64")

train_df,valid_df = df.copy()[:3], df.copy()[-2:]
data = TabularDataBunch.from_df(pth, train_df, valid_df, "val")

learn = get_tabular_learner(data, layers=[5,5],metrics=accuracy)
learn.fit(1,1e-2)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-99-e14bcf1c3a14> in <module>
----> 1 learn = get_tabular_learner(data, layers=[5,5],metrics=accuracy)
      2 learn.fit(1,1e-2)

~/miniconda3/envs/fastai/lib/python3.7/site-packages/fastai/tabular/data.py in get_tabular_learner(data, layers, emb_szs, metrics, ps, emb_drop, y_range, use_bn, **kwargs)
     95     emb_szs = data.get_emb_szs(ifnone(emb_szs, {}))
     96     model = TabularModel(emb_szs, len(data.cont_names), out_sz=data.c, layers=layers, ps=ps, emb_drop=emb_drop,
---> 97                          y_range=y_range, use_bn=use_bn)
     98     return Learner(data, model, metrics=metrics, **kwargs)
     99 

~/miniconda3/envs/fastai/lib/python3.7/site-packages/fastai/tabular/models.py in __init__(self, emb_szs, n_cont, out_sz, layers, ps, emb_drop, y_range, use_bn)
     20         layers = []
     21         for i,(n_in,n_out,dp,act) in enumerate(zip(sizes[:-1],sizes[1:],[0.]+ps,actns)):
---> 22             layers += bn_drop_lin(n_in, n_out, bn=use_bn and i!=0, p=dp, actn=act)
     23         self.layers = nn.Sequential(*layers)
     24 

~/miniconda3/envs/fastai/lib/python3.7/site-packages/fastai/layers.py in bn_drop_lin(n_in, n_out, bn, p, actn)
     31     layers = [nn.BatchNorm1d(n_in)] if bn else []
     32     if p != 0: layers.append(nn.Dropout(p))
---> 33     layers.append(nn.Linear(n_in, n_out))
     34     if actn is not None: layers.append(actn)
     35     return layers

~/miniconda3/envs/fastai/lib/python3.7/site-packages/torch/nn/modules/linear.py in __init__(self, in_features, out_features, bias)
     39         self.in_features = in_features
     40         self.out_features = out_features
---> 41         self.weight = Parameter(torch.Tensor(out_features, in_features))
     42         if bias:
     43             self.bias = Parameter(torch.Tensor(out_features))

TypeError: new() received an invalid combination of arguments - got (NoneType, int), but expected one of:
 * (torch.device device)
 * (torch.Storage storage)
 * (Tensor other)
 * (tuple of ints size, torch.device device)
      didn't match because some of the arguments have invalid types: (NoneType, int)
 * (object data, torch.device device)
      didn't match because some of the arguments have invalid types: (NoneType, int)

If you uncomment the #df[“val”]=(df[“o1”]>0.5).astype(“int64”), it changes the val from float64 to int64, basically a classification problem and the get_tabular_learner() method works fine. But when val is float64, or a regression problem, the get_tabular_learner() does not work.

1 Like
5

Same exact problem - trying to do a regression with my dependent variable having floating points and stumbling upon the same TypeError.

6

Right, looks like it doesn’t work when dependent variable is of type float.

From the rossman example notebooks, it seems like regression was used. Perhaps there’s an issue with fastai version 1.0.22?

Can anyone confirm previous versions that are working?

Thanks,

1 Like
7

After looking a the source code a little deeper, could this be the issue?

Below, data.c is not set when tasktype is TaskType.Regression. Could this be causing the issues when building the neural network layers?

def get_tabular_learner(data:DataBunch, layers:Collection[int], emb_szs:Dict[str,int]=None, metrics=None,
        ps:Collection[float]=None, emb_drop:float=0., y_range:OptRange=None, use_bn:bool=True, **kwargs):
    "Get a `Learner` using `data`, with `metrics`, including a `TabularModel` created using the remaining params."
    emb_szs = data.get_emb_szs(ifnone(emb_szs, {}))
 ---->>>>>   model = TabularModel(emb_szs, len(data.cont_names), out_sz=data.c, layers=layers, ps=ps, emb_drop=emb_drop,
                         y_range=y_range, use_bn=use_bn)
    return Learner(data, model, metrics=metrics, **kwargs)
8

I found a workaround, but I feel this should really be fixed in the DatasetBase constructor.

The DatasetBase constructor identifies the task type by calling self.get_task_type(). Thereafter, based on the values of classes and y, it should appropriately set self.c if not set already. It does not do this properly when task is of type TaskType.Regression as shown in the above example.

The workaround I believe is as follows: In the TabularDataBunch.from_df() call, add c=1.

Also remember for regression, the metrics function should be exp_rmspe and remember to cast your float values to np.float32. So, here is the above example with regression working when I add “c=1”.

Hope this helps.

from fastai import *
from fastai.tabular import * 

n=np.random.random((5,5)); n
array([[0.006474, 0.281819, 0.853687, 0.920277, 0.155631],
       [0.388049, 0.249157, 0.479755, 0.844395, 0.704095],
       [0.589191, 0.504232, 0.879636, 0.491535, 0.99221 ],
       [0.02966 , 0.923491, 0.103954, 0.360073, 0.516702],
       [0.963183, 0.665912, 0.259418, 0.8077  , 0.491568]])

df=pd.DataFrame(n,columns=['val','o1','o2','o3','o4'])

#df["val"]=(df["o1"]>0.5).astype("int64")
df=df.astype(np.float32)

train_df,valid_df = df.copy()[:3], df.copy()[-2:]
data = TabularDataBunch.from_df(".", train_df, valid_df, "val",c=1)

learn = get_tabular_learner(data, layers=[5,5],metrics=exp_rmspe)
learn.fit(1,1e-2)

Total time: 00:00
epoch  train_loss  valid_loss  exp_rmspe
1      0.338882    0.801665    0.591401   (00:00)
​
1 Like
9

Thank you so much! It worked!

Out of curiosity, what does option ‘c’ do? I can’t find anything on it in documentation.

10

I’m glad it helped. Based on reading the source code and if I understand correctly, I think c is an internal variable that represents the size of the last output layer.

1 Like
11

Based on memory, I believe the c attribute refers to the number of classes in a classification problem. For regression, the output should have one node in the final layer since we’re trying to predict a number, and that’s why setting data.c = 1 is a valid work-around.

12

Thanks guys – setting data.c = 1 works for me too.

=== Software === 
python version : 3.6.6
fastai version : 1.0.22
torch version  : 1.0.0.dev20181112
torch cuda ver 
torch cuda is  : **Not available** 

=== Hardware === 
No GPUs available 

=== Environment === 
platform       : Linux-4.4.0-1061-aws-x86_64-with-Ubuntu-18.04-bionic
distro         : #70-Ubuntu SMP Fri May 25 21:47:34 UTC 2018
conda env      : Unknown
python         : /home/lifetime-value/env/bin/python3
sys.path       : 
/home/lifetime-value/env/lib/python36.zip
/home/lifetime-value/env/lib/python3.6
/home/lifetime-value/env/lib/python3.6/lib-dynload
/usr/lib/python3.6
/home/lifetime-value/env/lib/python3.6/site-packages
/home/lifetime-value/env/lib/python3.6/site-packages/IPython/extensions
/root/.ipython
no supported gpus found on this system
13

So I use c=1 when creating TabularDataBunch because I would like to use it on a regression problem with bounded dependent variable. Docs for TabularModel say

The sizes of the blocks are given in layers and the probabilities of the Dropout in ps . The last size is out_sz , and we add a last activation that is a sigmoid rescaled to cover y_range (if it’s not None ).

I am looking for the same behavior with get_tabular_learner, i.e. a rescaled sigmoid in the last layer. Yet, when I print out my network’s structure, I do not see sigmoid anywhere. Instead, it looks like a structure for a classification problem:

Learner(data=<fastai.tabular.data.TabularDataBunch object at 0x7f08f1be8048>, model=TabularModel(
  (embeds): ModuleList(
    (0): Embedding(6, 3)
  )
  (emb_drop): Dropout(p=0.0)
  (bn_cont): BatchNorm1d(7, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (layers): Sequential(
    (0): Linear(in_features=10, out_features=200, bias=True)
    (1): ReLU(inplace)
    (2): BatchNorm1d(200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (3): Linear(in_features=200, out_features=100, bias=True)
    (4): ReLU(inplace)
    (5): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (6): Linear(in_features=100, out_features=1, bias=True)
  )
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99)), loss_func=MSELossFlat(), metrics=[<function rmse at 0x7f08f0db8ae8>], true_wd=True, bn_wd=True, wd=0.01, train_bn=True, path=PosixPath('data'), model_dir='models', callback_fns=[<class 'fastai.basic_train.Recorder'>], callbacks=[], layer_groups=[Sequential(
  (0): Embedding(6, 3)
  (1): Dropout(p=0.0)
  (2): BatchNorm1d(7, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (3): Linear(in_features=10, out_features=200, bias=True)
  (4): ReLU(inplace)
  (5): BatchNorm1d(200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (6): Linear(in_features=200, out_features=100, bias=True)
  (7): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (8): Linear(in_features=100, out_features=1, bias=True)
)])

I understand this is not the same object as get_tabular_learner, but I am sure they must be related. Do you know how to enable the rescaled sigmoid behavior?

UPDATE: I tried implementing ‘TabularModel’ and it doesn’t look like it includes the last bounded sigmoid layer:

TabularModel(
  (embeds): ModuleList(
    (0): Embedding(5, 3)
  )
  (emb_drop): Dropout(p=0.01)
  (bn_cont): BatchNorm1d(7, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (layers): Sequential(
    (0): Linear(in_features=10, out_features=200, bias=True)
    (1): ReLU(inplace)
    (2): BatchNorm1d(200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (3): Dropout(p=0.001)
    (4): Linear(in_features=200, out_features=100, bias=True)
    (5): ReLU(inplace)
    (6): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (7): Dropout(p=0.01)
    (8): Linear(in_features=100, out_features=1, bias=True)
  )
)
14

I think get_tabular_learner() can take an optional y_range parameter. Just pass it some two element tuple or list and it should work based on my reading of the code.

15

I did pass the range in both instances. I am just worried that I do not see any sigmoids in layers. Is it assumed?

EDIT: Never mind, I see it. I guess it is silent and is not reflected in layers printout. The code says:

if self.y_range is not None:
    x = (self.y_range[1]-self.y_range[0]) * F.sigmoid(x) + self.y_range[0]
return x
16

This no longer works in 1.0.24:

The network from get_tabular_learner continues doing a classification task even when ‘c’ is set to 1. My known values for the dependent variable are either whole numbers or whole numbers plus a half (e.g. 10, 3.5, -2.5, 0.5, etc.) and predictions give out values exactly either whole or whole plus a half with nothing else, e.g. 1.2345. I make predictions with learn.predict(row).

When I check layers from the following code (fastai v 1.0.24)

procs = [Categorify]
data = TabularDataBunch.from_df(path=path, 
                                df=df, 
                                dep_var=dep_var, 
                                valid_idx=valid_idx, 
                                procs=procs, 
                                cat_names=cat_col, 
                                cont_names=con_col)
emb_szs = {'weekday': 3}
learn = get_tabular_learner(data, layers=[200,100], emb_szs=emb_szs, y_range=y_range)

I get

[Sequential(
   (0): Embedding(6, 3)
   (1): Dropout(p=0.0)
   (2): BatchNorm1d(7, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
   (3): Linear(in_features=10, out_features=200, bias=True)
   (4): ReLU(inplace)
   (5): BatchNorm1d(200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
   (6): Linear(in_features=200, out_features=100, bias=True)
   (7): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
   (8): Linear(in_features=100, out_features=89, bias=True)
 )]

Layer 8 is seemingly performing a classification task with 89 classes…

UPDATE: If I switch to FastAI v 1.0.22 and the corresponding working code (TabularDataBunch.from_df needs different things, depending on FastAI version), the network performs regression, as before:

tfms = [Categorify]
data = TabularDataBunch.from_df(path, 
                                df_train, 
                                df_valid, 
                                dep_var, 
                                tfms=tfms, 
                                cat_names=cat_col, 
                                c=1)
emb_szs = {'weekday': 3}
# emb_szs = None
learn = get_tabular_learner(data, 
                            layers=[200,100], 
                            emb_szs=emb_szs, 
                            metrics=rmse, 
                            y_range=y_range)

[Sequential(
   (0): Embedding(6, 3)
   (1): Dropout(p=0.0)
   (2): BatchNorm1d(7, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
   (3): Linear(in_features=10, out_features=200, bias=True)
   (4): ReLU(inplace)
   (5): BatchNorm1d(200, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
   (6): Linear(in_features=200, out_features=100, bias=True)
   (7): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
   (8): Linear(in_features=100, out_features=1, bias=True)
 )]

I think I need to use v 1.0.24 as I cannot do predictions in v 1.0.22 on previously unseen data since predict method for the learner is only available in v 1.0.24…

2 Likes