Fastai2 Tabular Feature Importance

Hi all! I’ve been trying to figure out how to analyse feature importance for tabular data…There are a few relevant bits of code for Fastai1, but I’ve struggled to adapt them to the new code (I’m still pretty new at this!).

Eg: Feature importance in deep learning
Updated Feature Importance for Tabular?

I’ve had a dig through the documentation, and it looks like I might have to re-write this from scratch once I’ve figure it out (as you tend to iterate over your features and computer information loss)…or is there an easier way of doing this that I’ve missed?


1 Like

There’s a notebook, check out my tabular notebooks:

(almost anything and everything is in this repo and there are lectures to go along with it)


Oooo thank you! This is what I get for skipping instead of going through the damn book :stuck_out_tongue:

Actually this comes from my own course, not fastai’s book :wink: (See Walk with fastai2)

Hi @muellerzr , I am working with a tabular dataset and I would like to get the importance of each variable for each test observation.

On the GitHub you mention, which I found very interesting, it pulls the importance of the variables with respect to the whole DL model.

What do you think would be the way to go to get the most important variables for each observation in the test set?

Thank you very much in advance! :slight_smile:

1 Like

So, there’s a key difference in how permutation importance is done. What we do is permute the entire dataset and operate with this, shuffle all the rows around, and see how the results varied. “Technically” you could do this by having one row with a known test input, another that is completely noise, and have it run it. I’m not entirely sure how well that would work, but its one option. You can even still use the same function (I think), just have a df of two rows.

Edit: Thinking on this more, a change to that func would likely be needed, since we only have one labelled row. Specifically we would want to see the predicted answer for that row, and its variance. But you can use that func as a baseline for writing your own :smiley:

Otherwise there are better models suited for this stuff, specifically the ones that use attention. If you take a look at my TabNet notebook here, you can see that since attention is baked into the model, we can see individual feature importance in each row (and this is true for the Transformer-based tabular architectures as well):

Key Differences Between the Two:

  1. Permutation importance needs labeled data to understand how the results did
  2. TabNet and attention-based models do not
1 Like

Thank you very much for your quick response! I think it will be very useful for me to investigate the two options you suggest.

SHAP I think is another viable option too :slight_smile:


That’s a super interesting library! It looks like it solves @Saioa’s problem if I understand correctly.

I think you have finally convinced me to check out your course. Now I just have to find the time :sweat_smile: :sweat_smile: