From what I can tell the loss is getting set initially in the DataBunch class which is getting the value sometime when the train_ds is created
data.train_dl.dl.dataset.loss_func
returns the loss function
<function torch.nn.functional.cross_entropy(input, target, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='elementwise_mean')>
But I don’t see anywhere that that is actually passed. The closest thing I can find is this line in DataBunch which is the super for ImageDataBunch:
@property
def loss_func(self)->Dataset: return getattr(self.train_ds, 'loss_func', F.nll_loss)
So what this is saying is that if self.train_ds has a loss_func attribute, that it should use it, but if it doesn’t, it should use F.nll_loss which would look like this:
<function torch.nn.functional.nll_loss(input, target, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction='elementwise_mean')>
So at some point, self.train_ds is acquiring the loss_func attribute of cross entropy, but I don’t see anywhere that does that.
I am wondering if maybe it comes from here:
def __getattr__(self,k):
"Passthrough access to wrapped dataset attributes."
return getattr(self.ds, k)
from DatasetTfm, but that is just a hunch because it is passing through any attributes that are already associated to the attributes.
After the loss is set here though, it keeps using it going through model creation. Because the DataBunch has a loss already, the ConvLearner will just use that loss as defined here in the Learner code:
self.loss_func = ifnone(self.loss_func, self.data.loss_func)
So if ConvLearner.loss_func is None, then it will use the ConvLearner.data.loss_func which comes back to the original question. Where does the loss initially get set to cross entropy?