Tabular Data with Custom Multi-Target Model

Part 1 (2020)
1

I am trying to do a simple multi-target model. Here is a reproducible example:

import pandas as pd
import numpy as np
import torch.nn as nn
from fastai.tabular.data import TabularDataLoaders
from fastai.tabular.learner import TabularLearner

class Model(nn.Module):
    def __init__(self, num_features, num_targets):
        super().__init__()

        dropouts = [0.10, 0.20, 0.25, 0.25, 0.25]
        hidden_size = [384, 896, 896, 384]
        
        layers = [nn.BatchNorm1d(num_features)]
        in_size = num_features

        for i in range(len(hidden_size)):
            out_size = hidden_size[i]
            layers.append(nn.Dropout(dropouts[i]))
            layers.append(nn.Linear(in_size, out_size))
            layers.append(nn.BatchNorm1d(out_size))
            layers.append(nn.SiLU())  # SiLU aka swish
            in_size = out_size

        layers.append(nn.Dropout(dropouts[-1]))
        layers.append(nn.Linear(in_size, num_targets))
        layers.append(nn.Sigmoid())
        
        self.model = torch.nn.Sequential(*layers)

    def forward(self, x):
        x = self.model(x)
        return x

FEATURE_NAMES = [f'feature_{n}' for n in range(10)]
TARGET_NAMES = ['y1', 'y2']
train_data = pd.DataFrame(data=np.random.normal(size=(10000, 12)), columns=FEATURE_NAMES + TARGET_NAMES)

dls = TabularDataLoaders.from_df(train_data, cont_names=list(FEATURE_NAMES), y_names=TARGET_NAMES)

NUM_FEATURES = len(FEATURE_NAMES)
NUM_TARGETS = len(TARGET_NAMES)
net = Model(num_features=NUM_FEATURES, num_targets=NUM_TARGETS)

learner = TabularLearner(dls, model=net.double(), loss_func=nn.BCELoss())

When I call learner.fit(n_epoch=1) I get an error thrown

TypeError: forward() takes 2 positional arguments but 3 were given

What am I doing wrong here?

1 Like
2

fastai’s tabular dataloaders will always return two tensors (continuous and categorical). So in your case I would adjust your forward function like so:

    def forward(self, cat, cont):
        x = self.model(cont)
        return x
2 Likes
3

Fantastic. Works! Thank you so much!

4

If I may impose on you another basic question…

My learner does not seem to have all the special fastai methods. When I try, for example, learner.lr_find() I get

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-68-f01cf5c6afa7> in <module>
----> 1 learner.lr_find()

AttributeError: 'TabularLearner' object has no attribute 'lr_find'

The learner is just an instance of fastai TabularLearner. Shouldn’t it have all these methods? It does not have fit_one_cycle(...) either. I do have some other methods like fit and unfreeze.

I did spend an hour or so poking through the fastai GitHub code with no success. :slight_smile:

1 Like
5

When importing the fastai library you shouldn’t do direct imports, as it’s not meant to be utilized in that way. You should import with:

from fastai.tabular.all import *

For a more absolute import-based setup, you should see my comparison here in this lesson:

3 Likes
6

Ah, ok. Fascinating. I can’t bring my self to do an import * just yet :slight_smile:… so I will work through your course on this and new (to me) conventions that I suppose come with fastcore.

7

Not quite. This was present even with the old version of fastai. The idea is to ease imports and fastai will bring anything you need from all libraries (import pandas as pd, import torch, etc) so it’s all available in the namespace.

1 Like
8

If I want to use both cat and cont tensors, how can I combine them together in the forward function?

9

If I want to combine both cont and cat, should I just use torch.cat((cat, cont), 1)?