Use learned embeddings from tabular learner with a random forest

[rbunn80130 (Bob)]

I created and answered a similar question a few days ago, but then later realized I asked the wrong question.

Get activations of second to last layer in Tabular Model

I am training a tabular learner and would like to save the outputs of the next to last layer to train a random forest with (after training is complete). I did this before with a previous version of fast ai and the result was a significant accuracy boost. In my specific case my dataset was quite small and doing this was significantly better than using RF or NN alone. It appears that this should be simple after looking though the forums, but I can’t seem to figure out how to save these activati…

What I really need are the learned embeddings so I can replace the categorical columns with those values. This worked great for me previously, but not sure how to do this in the latest version of fastai. Are there any examples of this, or has anyone worked out the solution to this?

Thanks,

Bob

[Pak (Pavel)]

Accidentally I’ve described exactly that here

Some useful functions for tabular models

But I would like to start with something else (not exactly data interpretation). As I’ve managed to get the inner representation of the dataframe I’ve immediatily thought if I could use it as an input fo Random Forrest (to use embedding info in RF). So I’ve made a couple of functions that can do exactly that def emb_fwrd_sim(model, x_cat:Tensor, x_cont:Tensor)->Tensor: ''' Part that was completely taking from fastai Tabular model source Gets inner representation …

Hope it will help

[rbunn80130 (Bob)]

That’s awesome! Thanks!