RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #2 'target'

Part 1 (2019)
1

Hello all :smile:

I keep getting this error ‘RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #2 ‘target’’ each time I start training the learner, the source of the problem appears to be nll_loss function.

Here is the error:

RuntimeError Traceback (most recent call last)
in ()
----> 1 learn.fit_one_cycle(4)

C:\Users\D\Anaconda3\lib\site-packages\fastai\train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, wd, callbacks, **kwargs)
19 callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor,
20 pct_start=pct_start, **kwargs))
—> 21 learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
22
23 def lr_find(learn:Learner, start_lr:Floats=1e-7, end_lr:Floats=10, num_it:int=100, stop_div:bool=True, **kwargs:Any):

C:\Users\D\Anaconda3\lib\site-packages\fastai\basic_train.py in fit(self, epochs, lr, wd, callbacks)
164 callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
165 fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
–> 166 callbacks=self.callbacks+callbacks)
167
168 def create_opt(self, lr:Floats, wd:Floats=0.)->None:

C:\Users\D\Anaconda3\lib\site-packages\fastai\basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
92 except Exception as e:
93 exception = e
—> 94 raise e
95 finally: cb_handler.on_train_end(exception)
96

C:\Users\D\Anaconda3\lib\site-packages\fastai\basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
82 for xb,yb in progress_bar(data.train_dl, parent=pbar):
83 xb, yb = cb_handler.on_batch_begin(xb, yb)
—> 84 loss = loss_batch(model, xb, yb, loss_func, opt, cb_handler)
85 if cb_handler.on_batch_end(loss): break
86

C:\Users\D\Anaconda3\lib\site-packages\fastai\basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
20
21 if not loss_func: return to_detach(out), yb[0].detach()
—> 22 loss = loss_func(out, *yb)
23
24 if opt is not None:

C:\Users\D\Anaconda3\lib\site-packages\fastai\layers.py in call(self, input, target, **kwargs)
176 if self.floatify: target = target.float()
177 input = input.view(-1,input.shape[-1]) if self.is_2d else input.view(-1)
–> 178 return self.func.call(input, target.view(-1), **kwargs)
179
180 def CrossEntropyFlat(*args, axis:int=-1, **kwargs):

C:\Users\D\Anaconda3\lib\site-packages\torch\nn\modules\module.py in call(self, *input, **kwargs)
487 result = self._slow_forward(*input, **kwargs)
488 else:
–> 489 result = self.forward(*input, **kwargs)
490 for hook in self._forward_hooks.values():
491 hook_result = hook(self, input, result)

C:\Users\D\Anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)
902 def forward(self, input, target):
903 return F.cross_entropy(input, target, weight=self.weight,
–> 904 ignore_index=self.ignore_index, reduction=self.reduction)
905
906

C:\Users\D\Anaconda3\lib\site-packages\torch\nn\functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
1968 if size_average is not None or reduce is not None:
1969 reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 1970 return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
1971
1972

C:\Users\D\Anaconda3\lib\site-packages\torch\nn\functional.py in nll_loss(input, target, weight, size_average, ignore_index, reduce, reduction)
1788 .format(input.size(0), target.size(0)))
1789 if dim == 2:
-> 1790 ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
1791 elif dim == 4:
1792 ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)

RuntimeError: Expected object of scalar type Long but got scalar type Int for argument #2 ‘target’

Any ideas how to fix this ?

1 Like
2

Please post the code that creates the DataBunch and the learner. The cause of the error is likely to be found there. Then someone smarter than me will be able to help. :slightly_smiling_face:

3

Here it is :

> data = ImageDataBunch.from_folder(path, train=".", valid_pct=0.2,
>         ds_tfms=get_transforms(), size=224, num_workers=0).normalize(imagenet_stats)
> learn = create_cnn(data, models.resnet34, metrics=accuracy)
> learn.fit_one_cycle(4)
1 Like
4

Hi. I thought I might be able to help by looking at the preceding code, but was not able to. Perhaps someone with more knowledge will reply.

Speaking as a beginner, it looks like the the cross entropy loss function wants long targets but is getting ints. (These I believe should be the target class indexes.) I would look at data and data.one_batch() to see whether the inputs and targets are the correct shape and types. That might give you a clue as to why the labels are the wrong type.

HTH a little.

5

Yes it is apparently about the target classes being integers, I would like to know how to fix that from outside. Is there a way to change the target data type?

6

You can fix it by brute force. Define MyCrossEntropyFlat that simply converts target to longs and passes its parameters on to the original loss function. Then learn.loss_fn = MyCrossEntropyFlat. I have had to do such when fastai generated my targets as long rather than float, and I needed to get past the error.

However, as you point out, this is not the “right way”. The right way would be to discover why fastai is generating Ints, and fix either fastai or your use of it. But sometimes you have to use a workaround to move past an obstacle, and save the correct fix for a future day when we understand the library better. For what it’s worth, that’s my current approach to not getting stuck when the fix is not apparent.

But I do hope someone who understands this issue will post here. It would educate both me and Rawan.

1 Like
7

That’s a good Idea, I will give it a try ! ( I didn’t realize that I can specify the loss function :smile: maybe I should spend some time getting to know the documentation.)
Yes, I think the real fix should be from inside the fastai library (Although I am new to fastai, I can tell it is still under constant construction so these problems could happen.)

Thank you Malcolm.

8

Hi there ! Did you solve this problem, because I have the same.

I try to modify the loss by cast the tensor to a long, but an other error occurs

def nll_loss1(input, target, weight=None, size_average=None, ignore_index=-100,
         reduce=None, reduction='mean'):
# type: (Tensor, Tensor, Optional[Tensor], Optional[bool], int, Optional[bool], str) -> Tensor
r"""The negative log likelihood loss.

See :class:`~torch.nn.NLLLoss` for details.

Args:
    input: :math:`(N, C)` where `C = number of classes` or :math:`(N, C, H, W)`
        in case of 2D Loss, or :math:`(N, C, d_1, d_2, ..., d_K)` where :math:`K > 1`
        in the case of K-dimensional loss.
    target: :math:`(N)` where each value is :math:`0 \leq \text{targets}[i] \leq C-1`,
        or :math:`(N, d_1, d_2, ..., d_K)` where :math:`K \geq 1` for
        K-dimensional loss.
    weight (Tensor, optional): a manual rescaling weight given to each
        class. If given, has to be a Tensor of size `C`
    size_average (bool, optional): Deprecated (see :attr:`reduction`). By default,
        the losses are averaged over each loss element in the batch. Note that for
        some losses, there multiple elements per sample. If the field :attr:`size_average`
        is set to ``False``, the losses are instead summed for each minibatch. Ignored
        when reduce is ``False``. Default: ``True``
    ignore_index (int, optional): Specifies a target value that is ignored
        and does not contribute to the input gradient. When :attr:`size_average` is
        ``True``, the loss is averaged over non-ignored targets. Default: -100
    reduce (bool, optional): Deprecated (see :attr:`reduction`). By default, the
        losses are averaged or summed over observations for each minibatch depending
        on :attr:`size_average`. When :attr:`reduce` is ``False``, returns a loss per
        batch element instead and ignores :attr:`size_average`. Default: ``True``
    reduction (string, optional): Specifies the reduction to apply to the output:
        'none' | 'mean' | 'sum'. 'none': no reduction will be applied,
        'mean': the sum of the output will be divided by the number of
        elements in the output, 'sum': the output will be summed. Note: :attr:`size_average`
        and :attr:`reduce` are in the process of being deprecated, and in the meantime,
        specifying either of those two args will override :attr:`reduction`. Default: 'mean'

Example::

    >>> # input is of size N x C = 3 x 5
    >>> input = torch.randn(3, 5, requires_grad=True)
    >>> # each element in target has to have 0 <= value < C
    >>> target = torch.tensor([1, 0, 4])
    >>> output = F.nll_loss(F.log_softmax(input), target)
    >>> output.backward()
"""

target = target.long()



#target = target.type(torch.LongTensor)

if size_average is not None or reduce is not None:
    reduction = _Reduction.legacy_get_string(size_average, reduce)
dim = input.dim()
if dim < 2:
    raise ValueError('Expected 2 or more dimensions (got {})'.format(dim))

if input.size(0) != target.size(0):
    raise ValueError('Expected input batch_size ({}) to match target batch_size ({}).'
                     .format(input.size(0), target.size(0)))
if dim == 2:
    

    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
elif dim == 4:

    ret = torch._C._nn.nll_loss2d(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
else:
    # dim == 3 or dim > 4
    n = input.size(0)
    c = input.size(1)
    out_size = (n,) + input.size()[2:]
    if target.size()[1:] != input.size()[2:]:
        raise ValueError('Expected target size {}, got {}'.format(
            out_size, target.size()))
    input = input.contiguous().view(n, c, 1, -1)
    target = target.contiguous().view(n, 1, -1)
    reduction_enum = _Reduction.get_enum(reduction)
    if reduction != 'none':
        ret = torch._C._nn.nll_loss2d(
            input, target, weight, reduction_enum, ignore_index)
    else:
        out = torch._C._nn.nll_loss2d(
            input, target, weight, reduction_enum, ignore_index)
        ret = out.view(out_size)
return ret

learn = text_classifier_learner(data_clas, drop_mult=0.3 ,loss_func=[nll_loss1])

Now I have this error
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
5 #learn.get_preds(‘test.csv’ ) work
6
----> 7 learn.lr_find() # HERE IS THE BUG
8 learn.recorder.plot(skip_start=0)

D:\Programmes\Anaconda2\lib\site-packages\fastai\train.py in lr_find(learn, start_lr, end_lr, num_it, stop_div, **kwargs)
 29     cb = LRFinder(learn, start_lr, end_lr, num_it, stop_div)
 30     a = int(np.ceil(num_it/len(learn.data.train_dl)))
---> 31     learn.fit(a, start_lr, callbacks=[cb], **kwargs)
 32 
 33 def to_fp16(learn:Learner, loss_scale:float=512., flat_master:bool=False)->Learner:

D:\Programmes\Anaconda2\lib\site-packages\fastai\basic_train.py in fit(self, epochs, lr, wd, callbacks)
164         callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
165         fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
--> 166             callbacks=self.callbacks+callbacks)
167 
168     def create_opt(self, lr:Floats, wd:Floats=0.)->None:

D:\Programmes\Anaconda2\lib\site-packages\fastai\basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
 92     except Exception as e:
 93         exception = e
---> 94         raise e
 95     finally: cb_handler.on_train_end(exception)
 96 

D:\Programmes\Anaconda2\lib\site-packages\fastai\basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
 82             for xb,yb in progress_bar(data.train_dl, parent=pbar):
 83                 xb, yb = cb_handler.on_batch_begin(xb, yb)
---> 84                 loss = loss_batch(model, xb, yb, loss_func, opt, cb_handler)
 85                 if cb_handler.on_batch_end(loss): break
 86 

D:\Programmes\Anaconda2\lib\site-packages\fastai\basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
 20 
 21     if not loss_func: return to_detach(out), yb[0].detach()
---> 22     loss = loss_func(out, *yb)
 23 
 24     if opt is not None:

TypeError: 'list' object is not callable

Thanks for your help ! (Sorry for my english )

1 Like
9

I think your latter error is because there is a square bracket which made the loss function a list (learn = text_classifier_learner(data_clas, drop_mult=0.3 ,loss_func=[nll_loss1])) try deleting it and see what happens.
Regarding the error I got; I allowed myself to go and modify the fastai code and add the conversion to long from inside the FlattenedLoss() class in the “layers” file. (I am sure I shouldn’t do that though :laughing:), it solved the error but I got caught in a different one. However, I realized that there is a boolean variable called “floatify” in this class which controls making target a float, but I am not sure why it is needed and if it should be “longify” maybe :joy:.

1 Like
10

Ok thanks I try your solution and it’s working ! :wink:

1 Like
12

I faced the same problem with CrossEntropyLoss.
You can define your own loss function by inheriting the original PyTorch version and making a small change in the forward method.

Example:

class MyCrossEntropy(nn.CrossEntropyLoss):
  
  def forward(self, input, target):
    target = target.long()  # ADDED
    return F.cross_entropy(input, target, weight=self.weight, ignore_index=self.ignore_index, reduction=self.reduction)
2 Likes
13

Thanks for your solution. The fast ai lib must be converting my Longs to ints somewhere, grr.