I'm having trouble computing the metric of my model


I’m creating an NLP model with Huggingface Transformers. This model gives a score to a piece of text.

When I train the Huggingface trainer, it also computes predictions over the validation set. The validation accuracy is then calculated through the metric function you provide to the trainer when defining it.

I think what I’m having trouble with is how to use the Hugginface validation predictions variable. When I train my trainer, I get the following error message: TypeError: 'int' object is not callable.

This is my metric function, which computes RMSE:

def rmse(valid_pred):
    return torch.nn.functional.mse_loss(valid_pred.predictions, valid_pred.label_ids).sqrt()

valid_pred stores the predictions made by the trainer over the validation set.

Any input on how I should be using valid_pred or the metric function would be appreciated! You can view my notebook here if you wish: https://github.com/ForBo7/feedbackPrizeCompetition/blob/main/the_notebook.ipynb