Plot_top_losses() throwing "AttributeError: 'ImageList' object has no attribute 'cat_names'"

Part 1 (2019)
1

Hi everyone!
I am getting this error “AttributeError: ‘ImageList’ object has no attribute ‘cat_names’” when I execute interp.plot_top_losses(k = 5) on a ClassificationInterpretation object created from cnn learner. The learner was trained on an ImageDataBunch, which was created by first calling ImageList.from_df() and after splitting, labeling, converted to a databunch (and finally normalized).

Interestingly, the file where the error originates in is shown to be: <path_preix>/python3.7/site-packages/fastai/tabular/models.py; I am wondering why a function call on an ImageDataBunch would go to the tabular module. Does it look alright? Can someone help me understand how to fix the problem?

Here’s the code creating the databunch and the learner:

data = (ImageList.from_df(data_df, im_path, cols=‘path’)
.split_by_idxs(train_idxs, val_idxs)
.label_from_df(cols=‘class_label’)
.transform(get_transforms())
.databunch(bs=32)
).normalize(imagenet_stats)

learn = cnn_learner(data, models.resnet34, metrics=[error_rate, accuracy], callback_fns = learner_callbacks);

Here’s the complete error message:

AttributeError Traceback (most recent call last)
in
----> 1 interp.plot_top_losses(k=10)

~/anaconda3/envs/project_WACV/lib/python3.7/site-packages/fastai/tabular/models.py in _cl_int_plot_top_losses(self, k, largest, return_table)
53 tl_val, tl_idx = self.top_losses(k, largest)
54 classes = self.data.classes
—> 55 cat_names = self.data.x.cat_names
56 cont_names = self.data.x.cont_names
57 df = pd.DataFrame(columns=[[‘Prediction’, ‘Actual’, ‘Loss’, ‘Probability’] + cat_names + cont_names])

AttributeError: ‘ImageList’ object has no attribute ‘cat_names’

Here’s my install config:

=== Software ===
python : 3.7.3
fastai : 1.0.55
fastprogress : 0.1.21
torch : 1.1.0
nvidia driver : 410.104
torch cuda : 10.0.130 / is available
torch cudnn : 7501 / is enabled

=== Hardware ===
nvidia gpus : 1
torch devices : 1

  • gpu0 : 12192MB | TITAN X (Pascal)

=== Environment ===
platform : Linux-4.15.0-54-generic-x86_64-with-debian-buster-sid
distro : #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019

Cheers.
Tarik

4 Likes
2

No solution, but a comment and a few questions.

Note that the fact that the error originates in the tabular module is consistent with an error on a missing attribute cat_names: cat_names contains the names of the variables in a tabular data which you wish to consider as categorical.

Using grep, the only occurrence of plot_to_losses() is in the tabular.models module:

ClassificationInterpretation.plot_top_losses = _cl_int_plot_top_losses

and here is the source code to this _cl_int_plot_top_losses.

Now, there is another implementation of _cl_int_plot_top_losses in the vision.learner module. One solution seems to be to call

interp._cl_int_plot_top_losses(...)

instead of interp.plot_top_losses.

Other than that:

What is the line in the source code for the method plot_top_losses() where the error occurs?

Are you able to identify on which ImageList object the issue occurs?

For this I would break up the whole sequence in the data block API into intermediate steps, e.g.

img_list = ImageList.from_df(...)
item_lists = img_list.split_by_idxs(...)
lbl_lists = item_lists.label_from_df(...)
etc.
3

Thanks for replying!

Yeap, I noticed the other implementation of plot_top_losses (under vision/learner) is called correctly when I use the factory method to create a databunch from folder and use it to train a learner: ImageDataBunch.from_folder(…).normalize(imagenet_stats)

The error occurs on line#55:

cat_names = self.data.x.cat_names

of tabular.models.py

Unfortunately, breaking down the databunch generation sequence into imagelists doesn’t show which object the error originated from. I think the imagelist objects doesn’t exist separately inside a databunch, hence when the error pops up, it always has only one, anonymous imagelist object. Please correct me if I am wrong in assuming so.

Tarik

4

Come to think of it, I don’t think looking at your data bunch is relevant. It seems clearer that the problem is just with the call to the plot_top_losses() method.

Did you just try replacing

interp.plot_top_losses(k=5)

with

interp._cl_int_top_plot_losses(k=5)

?

Logically, this would call the “correct” method, i.e. the one in the vision.learner module.

1 Like
5

Thanks @Antoine.C

TL; DR:
problem root cause:

from fastai.vision import *
from fastai.tabular import *

problem fix (temporary):

from fastai.tabular import *
from fastai.vision import *

Long version: After tracing the code of creating interp from learner, I was able to find the problem and fix it. It lies in tabular.models.py, where the line:

ClassificationInterpretation.from_learner = _cl_int_from_learner

causes ClassificationInterpretation.from_learner() to be reassigned to the method from this class (it should point to vision.learner.py’s), causing this class’s plot_top_losses() method to be called when interp.plot_top_losses() is called. So it was the issue with importing fastai.tabular using * after importing fastai.vision with *. I swapped the order, and the problem got fixed. However, it is a temporary fix and I think should be corrected somewhere down the line.

Thanks for your help again, @Antoine.C!

Regards,

Tarik

2 Likes
6

@tarik Perhaps a guard or assert that checks the type of learner we pass into interpret? I was the one who put the PR in for plot_top_losses months ago for tabular to help bring it here, and didn’t know of this bug until now. My apologies! Please feel free to put in a or with how you believe we can fix it as I’m not entirely sure :frowning:

1 Like
7

@tarik Maybe you could create an issue? Or at least draw @jeremy or @sgugger’s attention to this?

Summary (from your last response):

Suppose you do the following imports:

from fastai.vision import *
from fastai.tabular import *

in this order.

If you create a vision learner and then want to create a ClassificationInterpretation from it, then the following line from the tabular.models module:

ClassificationInterpretation.from_learner = _cl_int_from_learner

causes it to be reassigned to the method from the tabular module.

As a result,

AttributeError: ‘ImageList’ object has no attribute ‘cat_names’

is thrown when executing

interp.plot_top_losses(k = 5)
8

No problem, @muellerzr, I gained a nonzero amount of knowledge out of this debugging experience, so all’s cool!

I think a wrapper in the ClassificationInterpretation class in fastai/train.py might help call the correct version of plot_top_losses() depending on the type of data associated with the learner that was used in creating the interpretation instance.

1 Like
9

@Antoine.C Sure. Will do.

10

Hi,

I am having exactly the same error when I try to add interp.plot_top_losses(9) as a line of code in the interpretation section of lesson 2 (in which there is not a fastai.tabular import, and hence the solutions here do not seem to apply).

Can anyone suggest how to resolve this?

Thanks!

1 Like
11

I also have the problem. Have you find the solution?
Thanks.

1 Like
12

@LinHan, I haven’t - about to post on the Lesson 2 discussion thread to see if someone there can offer a solution

13

I am also having this problem.

14

Thanks for replying.
I feel strange because I was able to use the method 20 days ago, in the same environment of kaggle kernel.
@jeremy Hope this can have Jeremy’s attention.
I am joining a algorithm competition regarding garbage classification and we are stopped in 89.68% acc. I am nearly sure that’s because of the dataset issue and I want to use plot_top_losses() to check the problem but get the same Error as above.

15

Think I know how to fix the issue the easiest way. We’ll just rename the tabular function like interp now has a default plot_multi_top_losses… I’ll put in a PR shortly

PR posted… Sorry about that guys! :frowning:

Sguggers comment, should be fixed today:

The issue is caused by importing fastai.widgets, which imports TabularDatabunch in tabular.data, which runs the code in models and replaces the monkey-patched plot_top_losses .

“I don’t like giving a different name for the same functionality and will solve the issue later this morning with a new tabular.learner module where the monkey-patch will happen without being touched by fastai.widgets .“

4 Likes
16

Sorry everyone, didn’t visit the forum in the last two months, hence didn’t notice the posts asking for help.

Good to know that a PR has been posted.