Forcing tabular_learner outputs

fastai
1

Hey all, first post!

I’ve got some data that I’ve been toying around with and the dependent variable is either class A, B, C,
or None.

When I build a tabular learner:

g = (TabularList.from_df(df=df, cat names=cat_variables, cont_names=cont_variables, procs=procs)
.split_by_rand_pct(valid_pct=0.2)
.label_from_df(cols=dep_variable)
.databunch( defaults=defaults.device, bs = 8196)

and:

learn. tabular_learner(g, layers=[10000,5000], ps=[0.001, 0.01], emb_drop=0.04, metrics=accuracy)

I obviously get accuracy/predictions that have 4 outputs: A, B, C, and None. It’s pretty accurate too, around 90%. However, my question is as follows: Is there a way to force tabular learner to only pick A or B? This works if I remove all C and None examples from the data, but I’d like to leave them in. They have value since this is a demographics based dataset, I just specifically want to avoid the scenario of the model predicting C or None.

Edit: I’ve got a lot of learning to do, so, I’m sorry if this is a dumb question!

2

You do it just how you described. If you wanted a case for c then you do rest vs C. A “tree” ish design to a bunch of tabular models. I’ve done this with my research. Sometimes it gives quite good results.

1 Like
3

Thanks so much for confirming my hunch on training multiple models!

I’m quite fortunate the size of my dataset is so big: 13 million. My C’s and None’s don’t total more than 2 million.

I was just hesitant because Jeremy seems big on don’t augment the data if you can avoid it.

4

@muellerzr

Kinda halfway related to this topic, do you know what happens to procs when you export a model and try single inference?

psuedocode:
newly_created_row = some_df_values #from a form input or something
learn = load_learner(some_exported_model.pkl)
learn.predict(newly_created_row)

edit: the relevant and not-clear-to-me part from the docs is: “As in the other applications, we just have to type learn.export() to save everything we’ll need for inference (here it includes the inner state of each processor)”

5

Procs are the procs of our training data. So FillMissing, Normalize, etc are for our original training data :slight_smile:

6

So, I need to rerun procs on an entirely new row that hasn’t been seen before?

7

Nope! Just learn.predict() with some dataframe row :slight_smile: think of procs the same as our image’s transforms (or processor) So they’re stored in our learner already

8

damn, I was hoping you wouldn’t have said that. I’m just getting really crappy predictions on things that are intuitively true and was hoping it was related to procs :cry:

9

How similar is your test data to your training data? Perhaps Maybe there’s data missing in columns that isn’t missing in your training data? Few things to try :slight_smile:

10

Dicts are not the same as dataframes.

:heart: Keep being awesome, dude @muellerzr .

1 Like