Transfer learning for tabular data - how to replace the head?

[muellerzr (Zachary Mueller)]

Searching the forums would have resulted in this post, with all the answers:

Tabular Transfer Learning and/or retraining with fastai

Hi @sgugger , I am happy to share with the community a basic demo of tabular transfer learning with fast.ai , thanks for pointing me in the direction of the fastai text. I am still unsure on how to handle the bias layers. I would really appreciate any help on how to modify the model architecture or layers (require_grad ?) to improve transfer accuracy , could you suggest me any paths of improvement ? @Jumonji : with you my version of the code; as you can see, I work directly from a pickled dicti…

TL:DR, transferring would really only be valuable at the embedding level, considering it the equivalent of our pretrained ResNets