Embedding weight initialization

Hi,

Can anyone tell me what’s the best way to initialize parameters in embeddings? I’m using random uniform <-1, 1>. Any papers discussing that?

Thanks in advance.

The default initializer in the TensorFlow embedding layer is
tf.truncated_normal_initializer with mean 0.0 and standard deviation 1/sqrt(sparse_id_column.length)

2 Likes

Thanks!