Fastai v2 chat

fastai fastai dev
1322

Hi,

I am facing this error when trying to use fastai2 in Kaggle. I have upgraded to torch 1.6.0 and installed fastai2 using pip. Would request help from anyone.

AttributeError Traceback (most recent call last)
in
7 from matplotlib import pyplot as plt
8 get_ipython().run_line_magic(‘matplotlib’, ‘inline’)
----> 9 from fastai2.torch_basics import *
10 from fastai2.basics import *
11 from fastai2.data.all import *

/opt/conda/lib/python3.7/site-packages/fastai2/torch_basics.py in
2 from .imports import *
3 from .torch_imports import *
----> 4 from .torch_core import *
5 from .layers import *

/opt/conda/lib/python3.7/site-packages/fastai2/torch_core.py in
244 # Cell
245 if not hasattr(torch,‘as_subclass’):
–> 246 setattr(torch, ‘as_subclass’, torch.Tensor.as_subclass)
247
248 # Cell

AttributeError: type object ‘Tensor’ has no attribute ‘as_subclass’

1323

I had the same error. Most likely you have PyTorch 1.5 installed (which is currently the default on Kaggle).

If you look at the release notes for PyTorch 1.6, the Tensor.as_subclass was added.

For me, running !pip install fastai2 in a kaggle notebook installs fastai2 with all the required dependencies (including PyTorch 1.6).

Note: In Kaggle notebooks, GPU acceleration in PyTorch doesn’t seem to work when upgrading PyTorch.

1 Like
1324

Thanks for your reply. I am still facing the issue after doing
!pip install fastai2

Even after running the above command, the
import torch
print(torch.__version__)

shows 1.5.1

1325

You need to install torch 1.6. Make sure if it gives you no GPU you install the right wheels for your cuda version

3 Likes
1326

Thanks a lot. I will try that.

1327

Thanks. This is why I couldn’t get cuda support on Kaggle with PyTorch 1.6. Kaggle is running cuda 10.1, while PyTorch assumes you’re using 10.2 when using the default pip install.

This works for me on Kaggle:

!pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
!pip install fastai2

Then when checking:

import torch
torch.cuda.is_available()

Then output is True, meaning that GPU acceleration is working :slight_smile:

5 Likes
1328

Looks like get_ipython is not defined in fastcore. It just creates some issues when using wandb in the fastai repo. It does not happen outside of the repo.
It is because wandb runs some git diff commands which also call nbdev_clean_nbs through the git filters settings.

Tried to do a PR adding in imports.py:

try: from IPython import get_ipython
except: pass

…but I get another error when trying a nbdev_clean_nbs:

$ nbdev_clean_nbs
Traceback (most recent call last):
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/bin/nbdev_clean_nbs", line 5, in <module>
    from nbdev.clean import nbdev_clean_nbs
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/lib/python3.8/site-packages/nbdev/__init__.py", line 6, in <module>
    from .flags import *
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/lib/python3.8/site-packages/nbdev/flags.py", line 115, in <module>
    for fn in fns: register_line_magic(fn)
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/lib/python3.8/site-packages/IPython/core/magic.py", line 229, in magic_deco
    raise NameError('Decorator can only run in context where '
NameError: Decorator can only run in context where `get_ipython` exists

EDIT: the issue has been resolved

1 Like
1329

Hey there!
I’m new with f
astaiv2 and had an application built on top of fastaiv1 Tabular.

I decided to upgrade my application in roder to have better support for interpretation (specially with @muellerzr 's fastinference)

Turns out i’m trying to implement wieghted sampling for trainning. I used to do it in Fastaiv1 like this:

        if sample_weights_col:
            sampler = torch.utils.data.sampler.WeightedRandomSampler(
                train_data[sample_weights_col].fillna(0),train_data.shape[0])

            # set sampler to train_dl
            db.train_dl = db.train_dl.new(shuffle=False, sampler=sampler)

Where sample_weights_cols is a column of the dataframe containing the sampling weights for each row of dataframe.

What i’ve been trying, with no succes is this in Fastai v2:

#create sample wieghts column
subsample['sample'] = (subsample['IVAMIRO'] == 'I7').astype(float)
#create sampler
sampler = torch.utils.data.sampler.WeightedRandomSampler(
            subsample[msk]['sample'].fillna(0).values,subsample[msk].shape[0])
#create tabular object
to = TabularPandas(subsample, procs, cat_names, cont_names, 
y_names=y_names, splits = splitter)
#create custom DLs for train and valid
dl_train = TabDataLoader(to.train, sampler = sampler , bs = 128, shuffle = False)
dl_valid = TabDataLoader(to.valid,bs = 64)

dataloaders = TabularDataLoaders(dl_train, dl_valid)

#sample from dl_train should return only rows where IVAMIRO == I7
dl_train.show_batch()

show_batch() returns this:
image

The samples should only contain rows where IVAMIRO == I7, and we can see its not the case. It seems like the sampler is not doing anything.

Does anyone know how to pass a custom sampler for a specific dataloader using fastai v2?

1330

If anyone is upgrading their windows 10 machines and is still on CUDA 9.2(like me!):

Current Version:

ver1
ver1914×201 11.1 KB

Following the instructions on PyTorch by downloading and running CUDA toolkit 11.0 Update 1 and then:

!pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html --user - Note I had to add --user at the end to prevent a Could not install packages due to an EnvironmentError: [WinError 5] Access is denied:

And then:
‘!pip install fastai2 --upgrade’

New install details:

ver2
ver2811×200 11.2 KB

1331

In v2, I’m having an issue with the new callback format.

loss_func = BCEWithLogitsLoss(pos_weight=weights)
acc_multi = partial(accuracy_multi, thresh=0.5)
learn = cnn_learner(data, resnet50, loss_func=loss_func, metrics=[acc_multi]).to_fp16()   callbacks = [
        SaveModelCallback(monitor='acc_multi', every_epoch=True, with_opt=True),
        TerminateOnNaNCallback(),
        EarlyStoppingCallback(monitor='valid_loss', min_delta=0.001, patience=10),
        ReduceLROnPlateau(monitor='valid_loss', min_delta=0.001, patience=5),
        MixUp(0.5)
    ]
    learn.fine_tune(20, 4.3e-2, freeze_epochs=4, cbs=callbacks)

The savemodel SaveModelCallback breaks on an assert trying to find the metric name. Am I making a mistake with how I am using the new callbacks, or is this a regression?

1332

I think it’s because it’s name should be “accuracy_multi” not “accu_multi”

1333

Yeah, I realized that almost immediately after posting it. didn’t realize that immediately because when I changed the name to the original one I had multilabel metrics in my callback list which also crash on distributed loads.

All working now! well, except multilabel metrics… but yeah.

1334

What’s the issue with to_fp16 with pytorch 1.6? I’m running with to_fp16 but I’m on pytorch 1.5. Should I just remain on 1.5 until the 1.6 issues for mixed precision is resolved?

1335

I found an issue around SEResNeXtBlock. For example with xse_resnext50, if either image dimension is not a multiple of 32 pixels, the forward method fails in layer 3. It is due to the fact that the rounding schema for the convpath and the idpath is different, thus self.convpath(x) and self.idpath(x) end up having different shape. Easy to repro with imagenette2 code. I am new to this forum, let me know if you want me to document it further. --Luc

1336

Just an FYI, it looks like some of the popular links are no longer working.

dev.fast.ai is no longer responding (server not found), for Welcome to fastai v2.

1337

With the release of fastai 2.0.0 and 2020 MOOC course, I think dev.fast.ai has been moved to docs.fast.ai.

Best regards,
Butch

1338

Putting this here so folks can find this in the future. Tonight @KevinB and I figured out how to not split your dataset while training:

 splitter=FuncSplitter(lambda x: False)

This will make all of your data go to the training DataLoader. Do note you’ll get a “Your generator is empty” warning when it tries to validate (but won’t show any errors)

Adding some other tags so folks can find this:

No validation set
Using only training data
Skip validation

7 Likes
1339

I spent so much time trying to make it work until I found this. Thanks @muellerzr

1341

Hi everyone,

I am trying to train a Text Classifier with a fine tuned Language Model using AWD_LSTM default architecture, and I have some questions.

  1. When I load a TextDataLoader, how can I do it to decode the elements of the dataset? So after Tokenize and Numericalize I get a tensor with numbers, how do I recover the original text from textdataloader.train_ds[0]? There is a method: textdataloader.decode but this does not give me back the text but rather a LMTensorText instance.

  2. Would not it be more comfortable, if we wanna do training with different schedules for different learning groups, to be able to pass a list of schedules to the scheduler. i.e. that we could write:

sched = {'lr': [SchedCos(lr/10.,lr) for lr in lrs]}

learner.fit(1, cbs=ParamScheduler(sched))

that is not allowed at the moment.

  1. Why the ActivationStats Callback just records statistics from the last layer of the AWD_LSTM default Language Model? When I type :
learner.activation_stats.stats[0]

gives me back:

[None,None,None,None,None,None,None,None,{'mean': -1.8462823629379272, 'std': 1.2596983909606934, 'near_zero': 0.9194439246831189, 'hist': tensor([1.2411e+06, 1.0021e+06, 8.0230e+05, 6.4066e+05, 5.0985e+05, 4.0494e+05,
        3.2018e+05, 2.5357e+05, 2.0129e+05, 1.5805e+05, 1.2631e+05, 1.0001e+05,
        8.0982e+04, 6.4612e+04, 5.2268e+04, 4.2857e+04, 3.4609e+04, 2.8632e+04,
        2.3751e+04, 1.9741e+04, 1.6459e+04, 1.3934e+04, 1.1901e+04, 1.0163e+04,
        8.5460e+03, 7.2550e+03, 6.3900e+03, 5.4860e+03, 4.8030e+03, 4.0000e+03,
        3.4060e+03, 2.9320e+03, 2.6210e+03, 2.2900e+03, 1.9420e+03, 1.6760e+03,
        1.4160e+03, 1.2110e+03, 1.0560e+03, 8.4500e+02])}]

So it is just tracking the statistics of the 9th layer.

Thanks a lot!!!

1342

Any updates on this?