I have pretrained a language model on a relatively small dataframe (~6k rows) with success. Perplexity values are quite low, which is representative of the type of semantic information involved.
Now when I try to load the encoder in order to create a pretrained supervised model for classification it seems to really drain all of my computer’s resources.
I don’t really mean that it’s filling up the GPU and training is slow or anything, but instead the whole computer seems to be very slow. Memory usage while training is around 16gb RAM which is really not an issue for my specs.
I was just wondering if anyone else noticed something similar and if you know of a way to debug this.
Thank you in advance.
Kind regards,
Theodore.