Hi,
Can anyone tell me what’s the best way to initialize parameters in embeddings? I’m using random uniform <-1, 1>. Any papers discussing that?
Thanks in advance.
Hi,
Can anyone tell me what’s the best way to initialize parameters in embeddings? I’m using random uniform <-1, 1>. Any papers discussing that?
Thanks in advance.
The default initializer in the TensorFlow embedding layer is
tf.truncated_normal_initializer with mean 0.0 and standard deviation 1/sqrt(sparse_id_column.length)
Thanks!