Good rule of thumb for embedding for categorical data

[StatisticDean]

Hi everyone, I’ve been using the tabular applications of fastai with good results since a few day. When you create a Tabular Learner, the default of fastai is to give you

min(600, round(1.6 * n_cat**0.56)) where n_cat is the number of values your categorical variable can take.

Are you familiar with other rules of thumb than can be used to choose a good embedding size?

[knesgood (Kyle Nesgood)]

This might be of interest.

Size of embedding for categorical variables

So it looks like it changed since I last looked. Previously, it was: def emb_sz_rule(n_cat:int)->int: return min(50, (n_cat//2)+1) Now it looks like it’s the following: def emb_sz_rule(n_cat:int)->int: return min(600, round(1.6 * n_cat**0.56)) I’m not sure of the reason why, but you can dig into the source code here: