I’m creating an NLP model with Huggingface Transformers. This model gives a score to a piece of text.
When I train the Huggingface trainer, it also computes predictions over the validation set. The validation accuracy is then calculated through the metric function you provide to the trainer when defining it.
I think what I’m having trouble with is how to use the Hugginface validation predictions variable. When I train my trainer, I get the following error message:
TypeError: 'int' object is not callable.
This is my metric function, which computes RMSE:
def rmse(valid_pred): return torch.nn.functional.mse_loss(valid_pred.predictions, valid_pred.label_ids).sqrt()
valid_pred stores the predictions made by the trainer over the validation set.
Any input on how I should be using
valid_pred or the metric function would be appreciated! You can view my notebook here if you wish: https://github.com/ForBo7/feedbackPrizeCompetition/blob/main/the_notebook.ipynb