Where is the loss function used for a learner defined?

Part 1 (2018)
1

Looking at the code and trying to figure out where the “loss” function is defined for a learner?

Specifically, I’m trying to change the loss function to be multi-class log loss … which in pytorch is, torch.nn.MultiLabelSoftMarginLoss I believe.

Also, is there a way to get the final loss on the validation dataset after the model has been trained?

1 Like
2

It’s defined for every kind of learner separately.
for ConvLearner it’s in conv_learner.py:

self.crit = F.binary_cross_entropy if data.is_multi else F.nll_loss
if data.is_reg: self.crit = F.l1_loss

In nlp.py You’’ find RNN_Learner:

self.crit = F.cross_entropy

and BOW_Learner:

self.crit = F.l1_loss

I didn’t find a way to change the loss, but you always can change the code of the library to give you the right loss.
Loss in Pytorch is often called criterion. So you can grep for ‘crit’ to find more usage of losses.

4 Likes
3

cool thanks @bushaev!

5

After you construct your learner, just type learn.crit = your_function_here .

9 Likes
6

Ah okay, cool. Thanks!

7

Hi,

I was trying to use the multilabel_margin_loss in dog breed dataset like this.

learn = ConvLearner.pretrained(arch, data, precompute=True)
learn.crit = F.multilabel_margin_loss

When I runlearn.fit I get this error.

HBox(children=(IntProgress(value=0, description='Epoch', max=8), HTML(value='')))
  0%|          | 0/325 [00:00<?, ?it/s]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-90-b3e0567589f3> in <module>()
----> 1 learn.fit(0.02, 8)

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in fit(self, lrs, n_cycle, wds, **kwargs)
    285         self.sched = None
    286         layer_opt = self.get_layer_opt(lrs, wds)
--> 287         return self.fit_gen(self.model, self.data, layer_opt, n_cycle, **kwargs)
    288 
    289     def warm_up(self, lr, wds=None):

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in fit_gen(self, model, data, layer_opt, n_cycle, cycle_len, cycle_mult, cycle_save_name, best_save_name, use_clr, use_clr_beta, metrics, callbacks, use_wd_sched, norm_wds, wds_sched_mult, use_swa, swa_start, swa_eval_freq, **kwargs)
    232             metrics=metrics, callbacks=callbacks, reg_fn=self.reg_fn, clip=self.clip, fp16=self.fp16,
    233             swa_model=self.swa_model if use_swa else None, swa_start=swa_start,
--> 234             swa_eval_freq=swa_eval_freq, **kwargs)
    235 
    236     def get_layer_groups(self): return self.models.get_layer_groups()

/usr/local/lib/python3.6/dist-packages/fastai/model.py in fit(model, data, n_epochs, opt, crit, metrics, callbacks, stepper, swa_model, swa_start, swa_eval_freq, **kwargs)
    127             batch_num += 1
    128             for cb in callbacks: cb.on_batch_begin()
--> 129             loss = model_stepper.step(V(x),V(y), epoch)
    130             avg_loss = avg_loss * avg_mom + loss * (1-avg_mom)
    131             debias_loss = avg_loss / (1 - avg_mom**batch_num)

/usr/local/lib/python3.6/dist-packages/fastai/model.py in step(self, xs, y, epoch)
     50         if self.fp16: self.m.zero_grad()
     51         else: self.opt.zero_grad()
---> 52         loss = raw_loss = self.crit(output, y)
     53         if self.loss_scale != 1: assert(self.fp16); loss = loss*self.loss_scale
     54         if self.reg_fn: loss = self.reg_fn(output, xtra, raw_loss)

RuntimeError: invalid argument 3: inconsistent target size at /pytorch/torch/lib/THCUNN/generic/MultiLabelMarginCriterion.cu:45

Could you tell me what I should do?

8

does this works on 1.0 ?