Lesson 5 discussion

[bennnun]

@marcohs
This is explained in this post:

Why do we divide the embedding by 3?

In the create_emd function, we divide emb by 3. I don’t recall the reasoning behind this – why is that? def create_emb(): n_fact = vecs.shape[1] emb = np.zeros((vocab_size, n_fact)) for i in range(1,len(emb)): word = idx2word[i] if word and re.match(r"^[a-zA-Z0-9\-]*$", word): src_idx = wordidx[word] emb[i] = vecs[src_idx] else: # If we can't find the word in glove, randomly initialize emb[i] = normal(scale=0.6, size=(n_fact,)) # This is our "rare w…