Hi @sgugger , I am happy to share with the community a basic demo of tabular transfer learning with fast.ai , thanks for pointing me in the direction of the fastai text. I am still unsure on how to handle the bias layers. I would really appreciate any help on how to modify the model architecture or layers (require_grad ?) to improve transfer accuracy , could you suggest me any paths of improvement ?
@Jumonji : with you my version of the code; as you can see, I work directly from a pickled dictionary instead of a CSV and only take care of embed weights, and not other layers yet.
You can see the model automatically starts at ~.30 loss instead of ~0.7, and everything runs+trains smoothly. I am ready to work on other problems, but I would first appreciate some feedback from anybody here !
CODE:
https://colab.research.google.com/drive/1yvA6pFPbmtwUUw1VDtPixoqWPTgkEfpM