001b_fit.ipynb discussion

fastai fastai dev
1

This thread is for discussing 001b_fit.ipynb.

2

In the early cell of the notebook, I get:

torch.tensor(x_train)

tensor([[0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.]])

which I found confusing - I thought there was something wrong with my config, and I tried to add:

torch.set_printoptions(precision=4)

to see no change, but then I discovered that it’s just because the data is mostly zeros.

Would it be useful to add an extra cell, like?

torch.tensor(x_train[0][150:200])

which returns:

tensor([0.0000, 0.0000, 0.0117, 0.0703, 0.0703, 0.0703, 0.4922, 0.5312, 0.6836,
        0.1016, 0.6484, 0.9961, 0.9648, 0.4961, 0.0000, 0.0000, 0.0000, 0.0000,
        0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1172,
        0.1406, 0.3672, 0.6016, 0.6641, 0.9883, 0.9883, 0.9883, 0.9883, 0.9883,
        0.8789, 0.6719, 0.9883, 0.9453, 0.7617, 0.2500, 0.0000, 0.0000, 0.0000,
        0.0000, 0.0000, 0.0000, 0.0000, 0.0000])

to show that there is a non-zero data and there is nothing wrong with the user’s config?

3

Yes it would :slight_smile:

1 Like
4

Thank you, @jeremy. I updated the notebook.

5

In this notebook, perhaps change default_device code to

default_device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

from just

default_device = torch.device('cuda')

so that for those running on non-cuda don’t get CUDA error in this and later notebooks

RuntimeError: Cannot initialize CUDA without ATen_cuda library. ...
3 Likes
6

Inside the notebook I was trying to Shift-TAB over loss.item() in the definition of loss_batch - it doesn’t work. It knows about opt variable methods, but not for the output of loss_fn.

loss_fn = F.cross_entropy
loss = loss_fn(model(x_valid[0:bs]), y_valid[0:bs])
loss
Out[68]:
tensor(2.3013, grad_fn=<NllLossBackward>)

item() is torch.Tensor.item()(doc).

I suppose this is because it returns a tensor object, whereas the actual class is a Tensor - do you see the mismatch?

The autocompletion and Shift-TAB do work for an explicit torch.Tensor.item(loss)

Any ideas how to get the notebook to resolve this situation? Perhaps we need to help it with some namespace aliasing? After running:

tensor = torch.Tensor

it did find the doc for loss.item() and autocompletion works.

But that probably is not it, since type(loss) does return torch.Tensor, where in print(loss) tensor is shown as type.

update: I tried to reproduce it again and wasn’t able to so I deleted this post. I came back to it later and again I couldn’t get the resolving to work, so I restored it as I guess there is some sanity left there.

It seems that if I replace in a later cell:

-       loss_fn(model(x_valid[0:bs]), y_valid[0:bs])
+  loss=loss_fn(model(x_valid[0:bs]), y_valid[0:bs])

and run it - then the notebook immediately knows of loss.item() docstring inside loss_batch, but with the current code it doesn’t. Very odd.

7

@stas jupyter does lookup for shift-tab and completion and stuff dynamically. There’s no parser or language server or anything. So inside a function def these things won’t work, since params to the function aren’t defined at that point.

1 Like
8

Will do.

1 Like