Not able to export learner, failing with `AttributeError: 'list' object has no attribute 'new_empty'`

JohnnyWobble · November 10, 2020, 9:43pm

Hi, I’m working an MNIST kaggle competition. I created the learner with

learner = cnn_learner(dls, resnet34, pretrained=False, loss_func=F.cross_entropy, metrics=accuracy)

and trained it twice using

learner.fit_one_cycle(1)
learner.fit_one_cycle(5)

and everything looked normal except it began to overfit at the end. But now when I call learner.export() it fails with

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-172-1fb6be3579a2> in <module>()
----> 1 learner.export()

3 frames
/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in __getattr__(self, k)
    133         if self._component_attr_filter(k):
    134             attr = getattr(self,self._default,None)
--> 135             if attr is not None: return getattr(attr,k)
    136         raise AttributeError(k)
    137     def __dir__(self): return custom_dir(self,self._dir())

AttributeError: 'list' object has no attribute 'new_empty'

I have no clue what is happening or where this list object is coming from.

Then also (maybe related) I can’t predict anything as well. If I try learner.predict(test_ds[0]), it just fails

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/fastai/learner.py in _with_events(self, f, event_type, ex, final)
    153     def _with_events(self, f, event_type, ex, final=noop):
--> 154         try:       self(f'before_{event_type}')       ;f()
    155         except ex: self(f'after_cancel_{event_type}')

27 frames
RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 3-dimensional input of size [3, 28, 28] instead

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
TypeError: expected Tensor as element 0 in argument 0, but got NoneType

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/fastcore/basics.py in range_of(a, b, step)
    428     "All indices of collection `a`, if `a` is a collection, otherwise `range`"
    429     if is_coll(a): a = len(a)
--> 430     return list(range(a,b,step) if step is not None else range(a,b) if b is not None else range(a))
    431 
    432 # Cell

TypeError: 'NoneType' object cannot be interpreted as an integer

stantonius · January 8, 2021, 7:35pm

Hey @JohnnyWobble did you ever get this sorted? This is the exact same error I am getting and am driving myself crazy - same cnn_learner as you, but export does not work and I get the following when predicting:

RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 3-dimensional input of size [3, 28, 28] instead

For what its worth, I am also training and predicting locally on Windows 10

stantonius · January 10, 2021, 1:54am

So I think I found a workaround on how to predict at least - it is a little bit patched together, but the predictions for the test set work

For anyone else that stumbles upon this post, maybe a summary will help first.

Custom Transforms were created to process the image data directly from the Kaggle Digit Recognizer competition DataFrame
As such the Mid-Level API was used to create the DataLoaders instead of ImageDataLoaders or DataBlock

In order to address the prediction errors that @JohnnyWobble mentioned, the following code worked to get predictions. Once the model was trained, I created the test dataset by returning a tuple of length 1 (this was key) from a TfmdLists that applies the same transforms (x_tfms) used to created the training data.

test_data = [(im,) for im in TfmdLists(test_data_raw, x_tfms)]

Then I assigned the new dataset to the Learner.dls

test_dl = learn.dls.test_dl(test_data, num_workers=0, shuffle=False)

Then to get the preds

preds = learn.get_preds(dl=test_dl)

But the output was still logits - so I had to take the max value and its index to get the meaningful predictions

results = torch.max(preds[0], dim=1)
results_df = pd.DataFrame([(i+1, x.item()) for i,x in enumerate(results.indices)], columns=["ImageId","Label"])

So while I was able to submit to the competition in the end, learn.export and learn.predict still do not work and return the errors above.

Any direction on how to export (and predict more easily) would be greatly appreciated.

JohnnyWobble · January 10, 2021, 5:46pm

Wow, thanks for the tip. I never got this working and I eventually gave up after hours of frustration. I was working on Google Colab (Linux), so I don’t think this is an OS issue. Although in the future, I would recommend not using Windows because Cuda is very funky.

kanastasiou · April 5, 2021, 11:22am

Similar issue, with similar work around. I was trying to export the LMModel7 at the end of the course. That’s a bit annoying.

hsaglani · December 28, 2021, 10:49am

Hi @jeremy , @muellerzr , Kindly help … export is not working … below are the details.

I am also facing same issue mentioned in this thread. When we do learn.export(), it gives ‘list’ object has no attribute ‘new_empty’
My notebook is here hars-mnist/harshad_mnist_classifier_jupyter.ipynb at main · harsag/hars-mnist · GitHub
I tried several tricks mentioned above and even pytorch code to export ONNX but was getting different errors. So could not get it working because I am not very good in low level code.
Also I think the way I have written my code, it has not populated data set object properly. I also tried datablock but same issue. Even for the code given in chapter 2, the export does not work.
So something in missing when we use linear layers.
Finally I did transfer learning on resnet18 using ImageDataLoaders and export worked. Please see hars-mnist/transfer_learning_colab.ipynb at main · harsag/hars-mnist · GitHub
I have made a nice app for complete MNIST hars-mnist/harshad_mnist_app1.ipynb at main · harsag/hars-mnist · GitHub but unfortunately I have to use the export coming from resnet18 (and not my own notebook). So please help

Also I take this opportunity to thank you all for this amazing course. Thanks a lot.

Please try my app here

bobseboy · March 25, 2022, 6:54am

Wasn’t able to come up with an actual solution. But one of two things at least solved my problem:

use .save() instead (yea, I know…)
In my case I only needed some components from the final object (e.g. the layers), so I grabbed the Pytorch components from the model e.g. “learner.rnn” and this I could export and/or save, inspired here: Export FastAI to PyTorch - #6 by muellerzr