TypeError: no implementation found for 'torch.nn.functional.cross_entropy'

Hi all,
I’m trying to run fastai on my ubuntu laptop.
I’ve got conda installed; and have:

(fastai) bgracey@SS:~$ pip show fastai
Name: fastai
Version: 2.1.10
....

(fastai) bgracey@SS:~$ pip show torch
Name: torch
Version: 1.7.1

The library has been working up till I hit, the below, any suggestions on what I could have wrong, or how do debug further would be most appreciated

I did find https://github.com/fastai/fastai/issues/2769 which sounds somewhat similar


TypeError                                 Traceback (most recent call last)
<ipython-input-67-99af2a5e6729> in <module>
----> 1 learn.fit_one_cycle(2, 0.01)

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/callback/schedule.py in fit_one_cycle(self, n_epoch, lr_max, div, div_final, pct_start, wd, moms, cbs, reset_opt)
    110     scheds = {'lr': combined_cos(pct_start, lr_max/div, lr_max, lr_max/div_final),
    111               'mom': combined_cos(pct_start, *(self.moms if moms is None else moms))}
--> 112     self.fit(n_epoch, cbs=ParamScheduler(scheds)+L(cbs), reset_opt=reset_opt, wd=wd)
    113 
    114 # Cell

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in fit(self, n_epoch, lr, wd, cbs, reset_opt)
    204             self.opt.set_hypers(lr=self.lr if lr is None else lr)
    205             self.n_epoch = n_epoch
--> 206             self._with_events(self._do_fit, 'fit', CancelFitException, self._end_cleanup)
    207 
    208     def _end_cleanup(self): self.dl,self.xb,self.yb,self.pred,self.loss = None,(None,),(None,),None,None

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153 
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _do_fit(self)
    195         for epoch in range(self.n_epoch):
    196             self.epoch=epoch
--> 197             self._with_events(self._do_epoch, 'epoch', CancelEpochException)
    198 
    199     def fit(self, n_epoch, lr=None, wd=None, cbs=None, reset_opt=False):

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153 
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _do_epoch(self)
    189 
    190     def _do_epoch(self):
--> 191         self._do_epoch_train()
    192         self._do_epoch_validate()
    193 

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _do_epoch_train(self)
    181     def _do_epoch_train(self):
    182         self.dl = self.dls.train
--> 183         self._with_events(self.all_batches, 'train', CancelTrainException)
    184 
    185     def _do_epoch_validate(self, ds_idx=1, dl=None):

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153 
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in all_batches(self)
    159     def all_batches(self):
    160         self.n_iter = len(self.dl)
--> 161         for o in enumerate(self.dl): self.one_batch(*o)
    162 
    163     def _do_one_batch(self):

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in one_batch(self, i, b)
    177         self.iter = i
    178         self._split(b)
--> 179         self._with_events(self._do_one_batch, 'batch', CancelBatchException)
    180 
    181     def _do_epoch_train(self):

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153 
    154     def _with_events(self, f, event_type, ex, final=noop):
--> 155         try:       self(f'before_{event_type}')       ;f()
    156         except ex: self(f'after_cancel_{event_type}')
    157         finally:   self(f'after_{event_type}')        ;final()

~/anaconda3/envs/fastai/lib/python3.8/site-packages/fastai/learner.py in _do_one_batch(self)
    164         self.pred = self.model(*self.xb)
    165         self('after_pred')
--> 166         if len(self.yb): self.loss = self.loss_func(self.pred, *self.yb)
    167         self('after_loss')
    168         if not self.training or not len(self.yb): return

~/anaconda3/envs/fastai/lib/python3.8/site-packages/torch/nn/functional.py in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction)
   2460         tens_ops = (input, target)
   2461         if any([type(t) is not Tensor for t in tens_ops]) and has_torch_function(tens_ops):
-> 2462             return handle_torch_function(
   2463                 cross_entropy, tens_ops, input, target, weight=weight,
   2464                 size_average=size_average, ignore_index=ignore_index, reduce=reduce,

~/anaconda3/envs/fastai/lib/python3.8/site-packages/torch/overrides.py in handle_torch_function(public_api, relevant_args, *args, **kwargs)
   1064 
   1065     func_name = '{}.{}'.format(public_api.__module__, public_api.__name__)
-> 1066     raise TypeError("no implementation found for '{}' on types that implement "
   1067                     '__torch_function__: {}'
   1068                     .format(func_name, list(map(type, overloaded_args))))

TypeError: no implementation found for 'torch.nn.functional.cross_entropy' on types that implement __torch_function__: [<class 'fastai.torch_core.TensorImageBW'>, <class 'fastai.torch_core.TensorCategory'>]

How are you defining your loss function? Also could you share your notebook by chance?

I’m working through - https://github.com/fastai/fastbook/blob/master/13_convolutions.ipynb
So really just trying to get something similar to run; any differences I have are accidental.

So that is using -
loss_func=F.cross_entropy

which has some level of definition in my local environment
<function torch.nn.functional.cross_entropy(input, target, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction=‘mean’)>

I uploaded the ipynb file here

It doesn’t start with !pip install -Uqq fastbook as I couldn’t get that to run in vscode jupyter; and I hope I did it correctly “manually” in the actual conda environment.

The notebook does run all the way up till fit_one_cycle - so the setup has some parts correctly available at least.

Just in case also try installing fastai via:
!pip install git+https://github.com/fastai/fastai
!pip install git+https://github.com/fastai/fastcore

This may have been fixed in master already

Yeah. 99% sure that was solved in master. You can either install it via that or replace it with CrossEntropyLossFlat until the next release comes out

Whoooop; indeed installing from github does resolve the issue! (And I tested in Colab; and hit the same issue there; I had such little faith in my local setup; I didnt try that before)

Thank you for the speedy response!!

Bonus questions if the answers are easily to hand:

Is it the patch that is referenced in https://github.com/fastai/fastai/issues/2769 that would have resolved it?
https://github.com/fastai/fastai/blob/8f8c629bd2da69a5d4b0d2dbb053ce5c0aa29180/fastai/torch_core.py#L324

From https://pypi.org/project/fastai/; it has Released: Dec 22, 2020 - so perhaps the fix is not the one above; and is more recent than 22nd Dec?

Thanks again!

It is. There was a comment on the 23rd in which Jeremy said he resolved it (in that issue). So for now it’s available in the dev version but not in the official release yet

Great; thanks for confirming

And thank you so much for the prompt responses; being able to resolve the issue and keep momentum is really really great!!!

1 Like

I am getting the same issue with the latest version as well. I am trying to fit on a complete MNIST dataset using the following:
learn = Learner(dls, simple_net, opt_func=SGD,
loss_func=F.cross_entropy, metrics=accuracy)

Installed fastai again, I am not getting the error now.