Feature Engineering - Target Encoding

Here is a great read on target encoding which something you would like to do when you have high cardinality in your categorical features. It is also desirable before feeding data into DNN s since it’s mapping each row to a number between (0,1).

1 Like

To clarify, this is isn’t for encoding the target (which is rarely used), but for encoding categorical independent variables based on the relationship between the variable and the target.

I am very suspicious of any paper that claims to provide an effective technique for machine learning, but shows no experiments whatsoever! If this technique results in better models, then the authors should show that for a number of datasets, and show experimentally where it works, and how much it helps, for what kinds of models.

Generally I haven’t seen techniques like this provide much if any improvement for tree based techniques, or for deep learning (which covers all the techniques that actually create the best models nowadays!)

For other types of model, this approach may be useful. An excellent package for preprocessing in R is described in this paper: https://arxiv.org/abs/1611.09477

2 Likes