Add name to a metric/loss defined as a module so SaveModelCallback can pick it up


I created a (several) losses by inheriting from nn.Module. The forward() method accepts predictions and targets. I added them to Learner ( metrics=[] parameter) and everything works decent. (Oh, I created them in this way because a linear combination of those losses act as an actual derivable loss)

I noticed that at the output I have sth like:

epoch train_loss	valid_loss	None	None	None	time
0	0.980436	1.023589	3.743178	0.343692	0.343692	00:14
1	11.154904	29.588293	146.756012	0.296344	0.296344	00:14

Now, those None in the names weren’t a problem until I wanted to have SaveModelCallback watching the 2’nd (or 3rd) metric. Afaik the SaveModelCallback accepts only named features.

I tried: * Inheriting from Metric too, I got some NotImplemented Exceptions, * writting by hand a .name parameter, * calling mk_metric(myLoss( . . .)) and adding that to metrics constructor parameter, and nothing worked.

My model and targets are of course non trivial so I need to do some tensor acrobatics before feeding them into classical F.mse_loss()

Close to re-implementing everything as a callback, what can I do? My goal is to make SaveModelCallback keep track of the n’th metric, I don’t really care about pretty printing the progress table. So either solution (adding a name to metric or specifying an index to SaveModel) is fine.

p.s. The link is “dead”, that is the #Creating-your-own-metric tag does not exist (3 Jan 2021).

Thank you and I hope I didn’t miss sth obvious on the forum/doc.fastai. But I’ve searched . . .