How can I save trained Pyrotch model but excluding word embeddings level (for NLP task) which takes most of the space. I want to recreate embeddings during inference stage and do not want to save them with the model.
This will not work because the rest of your weights depend on those values. Basically the embeddings are the first thing things flow through and if you remove them, all of the weights that occur after them will be using the wrong weights. You can chop off the last layer, but not the first. This is the same as freezing layers. If you have the first layer unfrozen, everything else has to be unfrozen as well or else your model will not be able to react appropriately when things start moving at the beginning.
It perfectly works in this case, since I was using pre-trained embeddings (from fastText or Glove) and they were frozen during the training phase. It was for a recent Kaggle competition and trying to unfreeze them during training did not help. So, in this case I just recreate embedding matrix (first layer) according to the words of the test set, using the pre-trained embeddings again.
And I found a technical way how to do it, not difficult. It was important for me to decrease size of saved models, since I wanted to train a number of models and make ensemble of them in the solution. And since also in the final testing phase I will get a new text.