What does emb_init do?

Just had been working through the FastAI code using columnar data? Just wondering what is the purpose of this emb_init?

Anyways, love the library, really awesome stuff.

  • Edwin

Hi. I think Jeremy covered this in Lesson 5 https://youtu.be/J99NV9Cr75I?t=46m2s

We do the first line just for convenience. We can write x instead of x.weight.data.

This means the third line is replaceing the x.weight.data with x.weight.data.uniform(-sc, sc). (also there is no not inplace version of .uniform)
Which essentially means filling in the weights of the embeddings with some initial values.
These values are generated exactly as np.random.uniform even the args that we had (min, max) are the same. Look at the numpy version’s documentation to see what kind of numbers it generates.

The second line is just getting the min and the max values for the uniform function. I don’t know what the sc stands for or why it is calculated this way… maybe someone else can help you with that.

It’s just an initializer for embeddings. As a rule of thumb it seems to work pretty well in my testing. I don’t have any rigorous basis for it however.

1 Like