Just had been working through the FastAI code using columnar data? Just wondering what is the purpose of this emb_init?

Anyways, love the library, really awesome stuff.

- Edwin

Just had been working through the FastAI code using columnar data? Just wondering what is the purpose of this emb_init?

Anyways, love the library, really awesome stuff.

- Edwin

We do the first line just for convenience. We can write `x`

instead of `x.weight.data`

.

This means the third line is replaceing the `x.weight.data`

with `x.weight.data.uniform(-sc, sc)`

. (also there is no not inplace version of .uniform)

Which essentially means filling in the weights of the embeddings with some initial values.

These values are generated exactly as `np.random.uniform`

even the args that we had (min, max) are the same. Look at the numpy version’s documentation to see what kind of numbers it generates.

The second line is just getting the min and the max values for the uniform function. I don’t know what the sc stands for or why it is calculated this way… maybe someone else can help you with that.

It’s just an initializer for embeddings. As a rule of thumb it seems to work pretty well in my testing. I don’t have any rigorous basis for it however.

1 Like