I was trying to use a Resnet to solve a multi-class task with a one-hot encoded dataset, and chose the nn.CrossEntropyLoss() function instead of the nn.BCEWithLogitLoss to avoid the sigmoid layer.
However, when I try tu use fastai’s metrics, I cannot choose the single-label metrics as they expect same-sized predictions and targets, and according to the docs I should use the multi-label metrics:
“Warning: All functions defined in this section are intended for single-label classification and targets that are not one-hot encoded. For multi-label problems or one-hot encoded targets, use the version suffixed with multi.”
However, the metrics like accuracy_multi and F1ScoreMulti are tailored for multi-label classifiers and not for multi-class. Even when I set sigmoid=False, they require a thresh, which does not apply to my type of problem.
What metrics should I use? Do I need to create custom metrics if I don’t wanna use integer labels instead of one-hot encoding?