Lesson 1 In-Class Discussion ✅

It could be possible that where you have the files is not accessible by your jupyter notebook. Try using path.ls() to locate your notebook and check if the path you’ve mentioned in your code is according to that or not.
Also like @kdorichev said, it’s group(1) :slight_smile:

1 Like
image=tensor2Images(x)

Any way for me to convert the image to databunch? or what function should I use? I had read the docs but most of the function requires a path and folder but I now have image object instead. Thanks

I have already answered this in my previous post. :wink:

Hello all,

I’ve just started with this course. After watching the first lesson, I’m planning to apply ResNet to a Kaggle dataset but the data give is in the form of pixel array and not in JPEG etc.
The (grayscale) images are of size 28x28 which have been flattened to a pixel array of length 784.
So, I just wanted to ask how should I process this data so that I can feed it into ResNet?

Thanks,
perceptron

Please take a look (Kaggle Notebook) how I did that.

@perceptron, Welcome to the community. And good luck!

2 Likes

@kdorichev Thank you so much! I would love to have a look and get back to you if I don’t get anything. :slight_smile:

Hi, the Lesson 1 files out-of-the-box does not work for me. Followed all instructions - fresh install / did not change anything (on Gradient / Paperspace). Really stuck:/ will appreciate help. Is it possible that something is outdated?

On step: interp.plot_top_losses(9, figsize=(15,11))

I get an error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-26-1b2e75ee979a> in <module>
      1 #interp.plot_top_losses(9, figsize=(7,6))
----> 2 interp.plot_top_losses(6, figsize=(15,11))

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/vision/learner.py in _cl_int_plot_top_losses(self, k, largest, figsize, heatmap, heatmap_thresh, alpha, cmap, show_text, return_fig)
    174     if show_text: fig.suptitle('Prediction/Actual/Loss/Probability', weight='bold', size=14)
    175     for i,idx in enumerate(tl_idx):
--> 176         im,cl = self.data.dl(self.ds_type).dataset[idx]
    177         cl = int(cl)
    178         title = f'{classes[self.pred_class[idx]]}/{classes[cl]} / {self.losses[idx]:.2f} / {self.preds[idx][cl]:.2f}' if show_text else None

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/data_block.py in __getitem__(self, idxs)
    647     def __getitem__(self,idxs:Union[int,np.ndarray])->'LabelList':
    648         "return a single (x, y) if `idxs` is an integer or a new `LabelList` object if `idxs` is a range."
--> 649         idxs = try_int(idxs)
    650         if isinstance(idxs, Integral):
    651             if self.item is None: x,y = self.x[idxs],self.y[idxs]

/opt/conda/envs/fastai/lib/python3.6/site-packages/fastai/torch_core.py in try_int(o)
    365     "Try to convert `o` to int, default to `o` if not possible."
    366     # NB: single-item rank-1 array/tensor can be converted to int, but we don't want to do this
--> 367     if isinstance(o, (np.ndarray,Tensor)): return o if o.ndim else int(o)
    368     if isinstance(o, collections.Sized) or getattr(o,'__array_interface__',False): return o
    369     try: return int(o)

AttributeError: 'Tensor' object has no attribute 'ndim'
2 Likes

This was fixed in pytorch 1.2. Please uprgade pytorch.

4 Likes

I am trying to apply the resnet34 model from lesson one to the old Kaggle competition for humpback whale tail identification. Given I don’t have a lot of background here I was wondering a few things

  1. There are 4000+ classes as each whale is individually labeled. Would the resnet network be suited for that large a number of classes? Does the number of classes make a difference here? When you have a large number of classes what kind of things would you do differently compared to having 30 or so like the pet example?
  2. All unidentified whales are labeled as “new_whale”. It seems like having such a class would cause issues as all the new whales are not necessarily related and would cause the class to become over eager. Should I throw out the new whales or is there a way to effectively use this data?

Thanks for any help!

I’ ve encountered exactly the same problem on Paperspace/Gradient with a Free-GPU notebook, but till now did not find a way to upgrade pytorch from version 1.0.0 (“quota exceeded…”). Thanks for any help!

2 Likes

I had the same problem, specifically on one of Paperspace’s Core Compute machines (not just a Gradient notebook).

The short story is, upgrade PyTorch specifically like this:

conda install pytorch torchvision cudatoolkit=10.0 -c pytorch

The long story is, if you just try to upgrade PyTorch, it will also upgrade the CUDA driver that will be incompatable with the NVIDIA GPU cards on their machines, disabling them from use in your notebooks. (I was getting this error when trying to debug why training was going extremely slow: The NVIDIA driver on your system is too old (found version 9010))

You can upgrade PyTorch, but make sure to flag it to use cudatoolkit=10.0, not 10.1.

3 Likes

Thx, sounds like a solution. On free Paperspace/Gradient I cannot however install anything as it exceeds 500 mb quota. Still working on it but appreciate your help.

UPDATE: Tom3 - if you chose lower tier free notebook na Paperspace/Gradient (I took P4000) you can install aquietlife’s command (seems they might have a different storage quota):
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
(from: Lesson 1 Discussion ✅)
then everything runs smooth. problem solved.

aquietlife - thanks once again

2 Likes

We should probably let someone know about this - the defaults from tutorial are not working. The installation package from tutorial should be modified I guess: https://course.fast.ai/start_gradient.html.

Anyone knows who to contact on this?

1 Like

As an alternative, you may want to try Google Colab. It has fairly new fastai and pytorch installed:

import fastai
fastai.__version__
'1.0.59'
import torch
torch.__version__
'1.3.1'

Glad it helped!! :slightly_smiling_face:

I contacted the support of Paperspace. Waiting for an answer. Thanks to all for help!

Thanks a lot! Will try.

Hi Jamesnixon94. I am a newbie myself and just got a dataset trained. Here’s what I did. It was a somewhat painful process of clicking around to figure it all out…

I uploaded a tarball of my images, then I opened up a terminal and ran the ‘tar -cvzxf yourfilename.tgz’ command to decompress it.

These instructions assume a familiarity with some command line navigating directories from the command line. It’s also not trivial to get all your images into the assumed label_imagenumber format into a directory…

1 Like

How do we exactly interpret train_loss, valid_loss and error_rate ? Is there a link where I can find definitions for these ?

I have used Jupyter notes havent installed GPU. In lesson 1 when i use learn.fit_one_cycle(4) its loading for ever. any inputs on this?