How to set and freeze pretrained embeddings?

I’ve some tabular data with several categorical variables. Each of these is embedded into space with different dimensions. For one of the categorical variables, I’d like to use a set of pretrained embeddings and keep them frozen during all the training. Is it possible to do so using the fastai library?

1 Like

I have the same issue. Trying to use a tabular learner to learn a categorical embedding based on something similiar to word2vec, but then I want to reuse this embedding in another model for transfer learning- there’s no easy (or even hard) way to do it as I can see without creating a whole mess of utility code. Isn’t this something fastai should support?

Please see the discussion here if you’d like to help with this Tabular Transfer Learning and/or retraining with fastai

Thanks - I just added a big post there of what I’ve done so far and my remaining issue. Hopefully we can figure this out together.