SOLVED - ImageCleaner causes AttributeError: ‘Tensor’ object has no attribute ‘ndim’

(Sebastian) #1

Hello, I have been running the Lesson2_download notebook on Paperspace and followed all the directions for setting up my environment.

When I get to the ‘Cleaning Up’ section in the lesson the cell fails to run and gives me a AttributeError: ‘Tensor’ object has no attribute ‘ndim’.

Here is the whole error message:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-27-1a5a092880e1> in <module>
----> 1 ImageCleaner(ds, idxs, path)
      2 # from tqdm.autonotebook import tqdm

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/widgets/image_cleaner.py in __init__(self, dataset, fns_idxs, path, batch_size, duplicates)
    210         self._csv_dict = {dataset.x.items[i]: dataset.y[i] for i in range(len(dataset))}
    211         self._deleted_fns:List[Path] = []
--> 212         self.next_batch()
    213 
    214     def make_payload(self, idx:int): return {'file_path': self._dataset.x.items[idx]}

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/widgets/image_cleaner.py in next_batch(self, _)
    193         "Fetches a next batch of images for rendering."
    194         if self.before_next_batch and hasattr(self, '_batch_payloads'): self.before_next_batch(self._batch_payloads)
--> 195         batch = tuple(islice(self._all_images, self.batch_size))
    196         self._batch_payloads = tuple(b.payload for b in batch)
    197         self.render(batch)

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/widgets/image_cleaner.py in create_image_list(self, fns_idxs, drop_batch_on_nonfile)
    172                 else chain.from_iterable(c for c in chunks(fns_idxs, self.batch_size)
    173                                            if all(Path(items[i]).is_file() for i in c)))
--> 174         for i in idxs: yield ImgData(self._dataset.x[i]._repr_jpeg_(), self._get_label(i), self.make_payload(i))
    175 
    176     def _get_label(self, idx):

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in __getitem__(self, idxs)
    117     def __getitem__(self,idxs:int)->Any:
    118         "returns a single item based if `idxs` is an integer or a new `ItemList` object if `idxs` is a range."
--> 119         idxs = try_int(idxs)
    120         if isinstance(idxs, Integral): return self.get(idxs)
    121         else: return self.new(self.items[idxs], inner_df=index_row(self.inner_df, idxs))

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/torch_core.py in try_int(o)
    365     "Try to convert `o` to int, default to `o` if not possible."
    366     # NB: single-item rank-1 array/tensor can be converted to int, but we don't want to do this
--> 367     if isinstance(o, (np.ndarray,Tensor)): return o if o.ndim else int(o)
    368     if isinstance(o, collections.Sized) or getattr(o,'__array_interface__',False): return o
    369     try: return int(o)

AttributeError: 'Tensor' object has no attribute 'ndim'

Does anyone know how to resolve the issue? I tried searching the forums and online but couldn’t seem to find anything related to the problem. I’ve tried making sure that I am in a notebook and not a lab.

3 Likes

#2

Hi Sebastian – I have the same issue. I have been working on a model this morning and am getting a nearly identical error. I pasted it below.

Also a newbie to the forum so I hope I am putting up appropriate information. I am on Gradient on the GPU free tier.

Mike


AttributeError Traceback (most recent call last)
in
2 # If you do run this on Jupyter Lab, you need to restart your runtime and
3 # runtime state including all local variables will be lost.
----> 4 ImageCleaner(ds, idxs, path)

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/widgets/image_cleaner.py in init(self, dataset, fns_idxs, path, batch_size, duplicates)
210 self._csv_dict = {dataset.x.items[i]: dataset.y[i] for i in range(len(dataset))}
211 self._deleted_fns:List[Path] = []
–> 212 self.next_batch()
213
214 def make_payload(self, idx:int): return {‘file_path’: self._dataset.x.items[idx]}

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/widgets/image_cleaner.py in next_batch(self, _)
193 “Fetches a next batch of images for rendering.”
194 if self.before_next_batch and hasattr(self, ‘_batch_payloads’): self.before_next_batch(self._batch_payloads)
–> 195 batch = tuple(islice(self._all_images, self.batch_size))
196 self._batch_payloads = tuple(b.payload for b in batch)
197 self.render(batch)

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/widgets/image_cleaner.py in create_image_list(self, fns_idxs, drop_batch_on_nonfile)
172 else chain.from_iterable(c for c in chunks(fns_idxs, self.batch_size)
173 if all(Path(items[i]).is_file() for i in c)))
–> 174 for i in idxs: yield ImgData(self._dataset.x[i].repr_jpeg(), self._get_label(i), self.make_payload(i))
175
176 def _get_label(self, idx):

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in getitem(self, idxs)
117 def getitem(self,idxs:int)->Any:
118 “returns a single item based if idxs is an integer or a new ItemList object if idxs is a range.”
–> 119 idxs = try_int(idxs)
120 if isinstance(idxs, Integral): return self.get(idxs)
121 else: return self.new(self.items[idxs], inner_df=index_row(self.inner_df, idxs))

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/torch_core.py in try_int(o)
365 “Try to convert o to int, default to o if not possible.”
366 # NB: single-item rank-1 array/tensor can be converted to int, but we don’t want to do this
–> 367 if isinstance(o, (np.ndarray,Tensor)): return o if o.ndim else int(o)
368 if isinstance(o, collections.Sized) or getattr(o,‘array_interface’,False): return o
369 try: return int(o)

AttributeError: ‘Tensor’ object has no attribute ‘ndim’

0 Likes

(David Nichols) #3

I had the same problem, using Azure VMs. A recent (Dec 15) change to the FastAI library introduced the reference to tensor.ndim, which only exists in PyTorch 1.3.0. So I had to upgrade pytorch and pytorchvision to make it all work again. I don’t have the VM running right now, but I did something like

sudo -i
conda -n fastai update pytorch # or maybe just ‘torch’
conda -n fastai update pytorchvision # or maybe just ‘torchvision’

Apparently, Python/Jupyter maintain many “environments”, which bind you to a set of package versions. The Azure VMs use the “fastai” environment for the course, so that’s the one to upgrade.

3 Likes

(David Nichols) #4

Oh yes, you can see which version of pytorch you have by importing it (generally, indirectly with “from fastai.vision import *”), and evaluating “torch.version”.

0 Likes

(David Nichols) #5

Argh, Markdown took out my underscores. The symbol “version” is preceded and followed by two underscore characters.

0 Likes

#6

Thanks for the advice. I am using Gradient and based on conda list my pytorch was out of date. . I removed pytorch and torchvision entirely, then ran:

conda install pytorch torchvision -c pytorch

That implemented multiple upgrades and the ImageCleaner is working now.

3 Likes

(Thon Deboer) #7

This no longer seems to work, since it will update to the CPU version of pytorch, which is not what we would likely want…

conda update -c fastai pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/tdeboer/anaconda3/envs/fastai

  added / updated specs:
- pytorch


The following NEW packages will be INSTALLED:

  _pytorch_select    pkgs/main/linux-64::_pytorch_select-0.1-cpu_0

The following packages will be UPDATED:

  pytorch            pytorch::pytorch-1.0.0-py3.7_cuda9.0.~ --> pkgs/main::pytorch-1.3.1-cpu_py37h62f834f_0
  torchvision        pytorch/noarch::torchvision-0.2.2-py_3 --> pkgs/main/linux-64::torchvision-0.4.2-cpu_py37h9ec355b_0


Proceed ([y]/n)? n

Does this mean I need to compile pytorch from source myself to support GPU?

UPDATE: I got pytorch 1.3.1 to work by doing this
conda install pytorch torchvision cudatoolkit=10.1 -c pytorch

that installed the gpu version of pytorch and fixed this issue with ndim issue that i saw in nbs-1

3 Likes

(Henry Bergström) #8

This worked for me. Thanks!

0 Likes

(Sebastian) #9

Hey everyone,

First, I want to say thank you to everyone who responded on this thread and helped out! I’ve been busy traveling so I have not had a chance to respond until now. Special thanks to @nichols who found the changes to the FastAI library from December 15th that referenced tensor.ndim, which only exists in Pytorch 1.3.0. Currently, the FastAI install downloads Pytorch 1.0.0.

SOLUTION

First, I removed and reinstalled the fastai library using conda to make sure I had all the latest packages. I opened a new terminal and ran the following code I got from this thread:

conda remove fastai && conda install -c fastai fastai

Next, I updated pytorch using slightly modified code that @mi239224 posted to also include removing the pytorch and torchvision libraries:

conda remove pytorch  torchvision && conda install -c pytorch  torchvision

This code uninstalled fastai-1.0.60-1, pytorch-1.0.0, and torchvision-0.2.1, and replaced them with newer installs of pytorch-1.3.1, torchvision-0.4.2, along with cudatoolkit-10.1.243.

Unfortunately, it uninstalled the fastai library, so I had to reinstall that in the end with conda install fastai -c fastai.

There probably is a better way to do everything without having to uninstall fastai, reinstall it, then reinstall it again at the end, but I was just trying to find a solution and move on. ImageCleaner now works with the solution above.

Hopefully this solves anybody else’s similar problems in the future. Again, thank you to everyone who responded and helped me find a solution! :smiling_face_with_three_hearts:

1 Like

(Jacques) #10

Good temporary solution. Worked well, thanks!

0 Likes