Transfer learning with tabular data

Part 1 2022
1

Wonder if there is any interesting work done around Transfer learning and tabular data?

I was thinking why not use same ideas in ULMFit where you would:

  1. train an encoder-decoder on a large tabular dataset with goal to reconstruct the dataset
  2. then on similar dataset (same columns/types) but will few rows and an additional label column, you could use the encoder as backbone to encode the features and add a head to learn the label column

For instance, part 1 could be trained on entire historical data but part 2 only on a recent subset with labels (either classification or regression).

This is just an idea and I’m not sure if it’s good or not.

3 Likes
2

Would recommend reading this post in full:

4 Likes
3

ok so the idea is to use embedding of individual columns