I am trying to classify some text articles as low, medium or high quality however the model is useless and training it is making it worse rather than improving.
While training my language model it starts out very accurate which is surprising since the articles are quite technical however once unfreezing the model and training the whole model the accuracy gets worse as well as the training and validation loss.
The plot of losses is also very confusing.
This training was taking a very long time so I decided to try and classify the documents to see if anything would happen. I used 1e-1 for the learning rate based off this plot.
After starting to train my classifier I got a terrible result with no idea what I am doing wrong.