Regression using ULMfit


(leon yin) #1

I tried training the classifier bit of ULMfit for a regression task by changing c=1, and changing the loss criterion, but I still get errors stemming from the accuracy metric function called from model.py.

Is there something I am overlooking, or does new code need to be written for regression tasks?
I am happy to submit a PR and write some code if we want to build a get_rnn_regression function.

Background I am trying to calculate political ideology (a scale that can be normalized from 0-1) from a transcript.

For reference here is the traceback:
TypeError Traceback (most recent call last)
in ()
----> 1 learn.fit(lrs, 1, wds=wd, cycle_len=1, use_clr=(8,3))

/home/ly501/anaconda3/lib/python3.6/site-packages/fastai/learner.py in fit(self, lrs, n_cycle, wds, **kwargs)
    285         self.sched = None
    286         layer_opt = self.get_layer_opt(lrs, wds)
--> 287         return self.fit_gen(self.model, self.data, layer_opt, n_cycle, **kwargs)
    288 
    289     def warm_up(self, lr, wds=None):

/home/ly501/anaconda3/lib/python3.6/site-packages/fastai/learner.py in fit_gen(self, model, data, layer_opt, n_cycle, cycle_len, cycle_mult, cycle_save_name, best_save_name, use_clr, use_clr_beta, metrics, callbacks, use_wd_sched, norm_wds, wds_sched_mult, use_swa, swa_start, swa_eval_freq, **kwargs)
    232             metrics=metrics, callbacks=callbacks, reg_fn=self.reg_fn, clip=self.clip, fp16=self.fp16,
    233             swa_model=self.swa_model if use_swa else None, swa_start=swa_start,
--> 234             swa_eval_freq=swa_eval_freq, **kwargs)
    235 
    236     def get_layer_groups(self): return self.models.get_layer_groups()

/home/ly501/anaconda3/lib/python3.6/site-packages/fastai/model.py in fit(model, data, n_epochs, opt, crit, metrics, callbacks, stepper, swa_model, swa_start, swa_eval_freq, **kwargs)
    148 
    149         if not all_val:
--> 150             vals = validate(model_stepper, cur_data.val_dl, metrics)
    151             stop=False
    152             for cb in callbacks: stop = stop or cb.on_epoch_end(vals)

/home/ly501/anaconda3/lib/python3.6/site-packages/fastai/model.py in validate(stepper, dl, metrics)
    209             else: batch_cnts.append(len(x))
    210             loss.append(to_np(l))
--> 211             res.append([f(preds.data, y) for f in metrics])
    212     return [np.average(loss, 0, weights=batch_cnts)] + list(np.average(np.stack(res), 0, weights=batch_cnts))
    213 

/home/ly501/anaconda3/lib/python3.6/site-packages/fastai/model.py in <listcomp>(.0)
    209             else: batch_cnts.append(len(x))
    210             loss.append(to_np(l))
--> 211             res.append([f(preds.data, y) for f in metrics])
    212     return [np.average(loss, 0, weights=batch_cnts)] + list(np.average(np.stack(res), 0, weights=batch_cnts))
    213 

/home/ly501/anaconda3/lib/python3.6/site-packages/fastai/metrics.py in accuracy(preds, targs)
      8 def accuracy(preds, targs):
      9     preds = torch.max(preds, dim=1)[1]
---> 10     return (preds==targs).float().mean()
     11 
     12 def accuracy_thresh(thresh):

/home/ly501/anaconda3/lib/python3.6/site-packages/torch/tensor.py in __eq__(self, other)
    358 
    359     def __eq__(self, other):
--> 360         return self.eq(other)
    361 
    362     def __ne__(self, other):

TypeError: eq received an invalid combination of arguments - got (torch.cuda.FloatTensor), but expected one of:
 * (int value)
      didn't match because some of the arguments have invalid types: (!torch.cuda.FloatTensor!)
 * (torch.cuda.LongTensor other)
      didn't match because some of the arguments have invalid types: (!torch.cuda.FloatTensor!)

#2

Hi, you need to pass learn.crit = F.mse_loss for regression. By default RNN_Learner uses F.cross_entropy.


ULMFit for sequence tagging
(urmas pitsi) #3

try this:

  1. print out the types to see which one is causing problems:
    metrics.py, before line 10: print(type(preds), type(targs))

  2. if targs = cuda.long.tensor then
    try to convert ‘y’ (targets) in dataset from ‘int’ to ‘float’. Then it should work.
    you can try this:
    data.trn_ds.y = (data.trn_ds.y).astype(np.float32)
    data.val_ds.y = (data.val_ds.y).astype(np.float32)


(leon yin) #4

Thanks for reply, I made that change, but I also had to change learn.metrics

learn.crit = torch.nn.MSELoss()
learn.metrics = [rmse]

where rmse is

def rmse(preds, targs):
    """Compute root mean squared error"""
    return torch.sqrt(torch.mean((targs - preds).pow(2)))

Let’s see what happens now


(Nafiz Hamid) #5

Did it work for you? I am trying to do regression also but getting errors.


(leon yin) #6

What are the errors?
I made a modest change from torch.sqrt to np.sqrt, and it works fine now:

def rmse(preds, targs):
    '''Compute root mean squared error'''
    return np.sqrt(torch.mean((targs - preds).pow(2)))