Fastai v2 chat

andreasl · August 13, 2020, 9:45am

I had the same error. Most likely you have PyTorch 1.5 installed (which is currently the default on Kaggle).

If you look at the release notes for PyTorch 1.6, the Tensor.as_subclass was added.

For me, running !pip install fastai2 in a kaggle notebook installs fastai2 with all the required dependencies (including PyTorch 1.6).

Note: In Kaggle notebooks, GPU acceleration in PyTorch doesn’t seem to work when upgrading PyTorch.

Aninda · August 13, 2020, 10:48am

Thanks for your reply. I am still facing the issue after doing
!pip install fastai2

Even after running the above command, the
import torch
print(torch.__version__)

shows 1.5.1

muellerzr · August 13, 2020, 11:41am

You need to install torch 1.6. Make sure if it gives you no GPU you install the right wheels for your cuda version

Aninda · August 13, 2020, 11:42am

Thanks a lot. I will try that.

andreasl · August 13, 2020, 2:11pm

Thanks. This is why I couldn’t get cuda support on Kaggle with PyTorch 1.6. Kaggle is running cuda 10.1, while PyTorch assumes you’re using 10.2 when using the default pip install.

This works for me on Kaggle:

!pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
!pip install fastai2

Then when checking:

import torch
torch.cuda.is_available()

Then output is True, meaning that GPU acceleration is working

boris · August 16, 2020, 10:44pm

Looks like get_ipython is not defined in fastcore. It just creates some issues when using wandb in the fastai repo. It does not happen outside of the repo.
It is because wandb runs some git diff commands which also call nbdev_clean_nbs through the git filters settings.

Tried to do a PR adding in imports.py:

try: from IPython import get_ipython
except: pass

…but I get another error when trying a nbdev_clean_nbs:

$ nbdev_clean_nbs
Traceback (most recent call last):
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/bin/nbdev_clean_nbs", line 5, in <module>
    from nbdev.clean import nbdev_clean_nbs
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/lib/python3.8/site-packages/nbdev/__init__.py", line 6, in <module>
    from .flags import *
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/lib/python3.8/site-packages/nbdev/flags.py", line 115, in <module>
    for fn in fns: register_line_magic(fn)
  File "/home/boris/.local/share/virtualenvs/fastai2-MNobJKnX/lib/python3.8/site-packages/IPython/core/magic.py", line 229, in magic_deco
    raise NameError('Decorator can only run in context where '
NameError: Decorator can only run in context where `get_ipython` exists

EDIT: the issue has been resolved

alan_ganem · August 17, 2020, 5:11pm

Hey there!
I’m new with f
astaiv2 and had an application built on top of fastaiv1 Tabular.

I decided to upgrade my application in roder to have better support for interpretation (specially with @muellerzr 's fastinference)

Turns out i’m trying to implement wieghted sampling for trainning. I used to do it in Fastaiv1 like this:

        if sample_weights_col:
            sampler = torch.utils.data.sampler.WeightedRandomSampler(
                train_data[sample_weights_col].fillna(0),train_data.shape[0])

            # set sampler to train_dl
            db.train_dl = db.train_dl.new(shuffle=False, sampler=sampler)

Where sample_weights_cols is a column of the dataframe containing the sampling weights for each row of dataframe.

What i’ve been trying, with no succes is this in Fastai v2:

#create sample wieghts column
subsample['sample'] = (subsample['IVAMIRO'] == 'I7').astype(float)
#create sampler
sampler = torch.utils.data.sampler.WeightedRandomSampler(
            subsample[msk]['sample'].fillna(0).values,subsample[msk].shape[0])
#create tabular object
to = TabularPandas(subsample, procs, cat_names, cont_names, 
y_names=y_names, splits = splitter)
#create custom DLs for train and valid
dl_train = TabDataLoader(to.train, sampler = sampler , bs = 128, shuffle = False)
dl_valid = TabDataLoader(to.valid,bs = 64)

dataloaders = TabularDataLoaders(dl_train, dl_valid)

#sample from dl_train should return only rows where IVAMIRO == I7
dl_train.show_batch()

show_batch() returns this:

The samples should only contain rows where IVAMIRO == I7, and we can see its not the case. It seems like the sampler is not doing anything.

Does anyone know how to pass a custom sampler for a specific dataloader using fastai v2?

amritv · August 18, 2020, 5:24pm

If anyone is upgrading their windows 10 machines and is still on CUDA 9.2(like me!):

Current Version:

Following the instructions on PyTorch by downloading and running CUDA toolkit 11.0 Update 1 and then:

!pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html --user - Note I had to add --user at the end to prevent a Could not install packages due to an EnvironmentError: [WinError 5] Access is denied:

And then:
‘!pip install fastai2 --upgrade’

New install details:

FuzzyBabyDucks · August 18, 2020, 7:07pm

In v2, I’m having an issue with the new callback format.

loss_func = BCEWithLogitsLoss(pos_weight=weights)
acc_multi = partial(accuracy_multi, thresh=0.5)
learn = cnn_learner(data, resnet50, loss_func=loss_func, metrics=[acc_multi]).to_fp16()   callbacks = [
        SaveModelCallback(monitor='acc_multi', every_epoch=True, with_opt=True),
        TerminateOnNaNCallback(),
        EarlyStoppingCallback(monitor='valid_loss', min_delta=0.001, patience=10),
        ReduceLROnPlateau(monitor='valid_loss', min_delta=0.001, patience=5),
        MixUp(0.5)
    ]
    learn.fine_tune(20, 4.3e-2, freeze_epochs=4, cbs=callbacks)

The savemodel SaveModelCallback breaks on an assert trying to find the metric name. Am I making a mistake with how I am using the new callbacks, or is this a regression?

muellerzr · August 18, 2020, 8:52pm

I think it’s because it’s name should be “accuracy_multi” not “accu_multi”

FuzzyBabyDucks · August 19, 2020, 6:54pm

Yeah, I realized that almost immediately after posting it. didn’t realize that immediately because when I changed the name to the original one I had multilabel metrics in my callback list which also crash on distributed loads.

All working now! well, except multilabel metrics… but yeah.

ai_padawan · August 24, 2020, 5:40pm

What’s the issue with to_fp16 with pytorch 1.6? I’m running with to_fp16 but I’m on pytorch 1.5. Should I just remain on 1.5 until the 1.6 issues for mixed precision is resolved?

Luc · September 2, 2020, 10:12pm

I found an issue around SEResNeXtBlock. For example with xse_resnext50, if either image dimension is not a multiple of 32 pixels, the forward method fails in layer 3. It is due to the fact that the rounding schema for the convpath and the idpath is different, thus self.convpath(x) and self.idpath(x) end up having different shape. Easy to repro with imagenette2 code. I am new to this forum, let me know if you want me to document it further. --Luc

than3 · September 3, 2020, 12:22pm

Just an FYI, it looks like some of the popular links are no longer working.

dev.fast.ai is no longer responding (server not found), for Welcome to fastai v2.

butchland · September 3, 2020, 1:34pm

With the release of fastai 2.0.0 and 2020 MOOC course, I think dev.fast.ai has been moved to docs.fast.ai.

Best regards,
Butch

muellerzr · September 5, 2020, 1:37am

Putting this here so folks can find this in the future. Tonight @KevinB and I figured out how to not split your dataset while training:

 splitter=FuncSplitter(lambda x: False)

This will make all of your data go to the training DataLoader. Do note you’ll get a “Your generator is empty” warning when it tries to validate (but won’t show any errors)

Adding some other tags so folks can find this:

No validation set
Using only training data
Skip validation

NavneetSajwan · September 12, 2020, 6:54am

I spent so much time trying to make it work until I found this. Thanks @muellerzr

arnau · September 28, 2020, 2:01pm

Hi everyone,

I am trying to train a Text Classifier with a fine tuned Language Model using AWD_LSTM default architecture, and I have some questions.

When I load a TextDataLoader, how can I do it to decode the elements of the dataset? So after Tokenize and Numericalize I get a tensor with numbers, how do I recover the original text from textdataloader.train_ds[0]? There is a method: textdataloader.decode but this does not give me back the text but rather a LMTensorText instance.
Would not it be more comfortable, if we wanna do training with different schedules for different learning groups, to be able to pass a list of schedules to the scheduler. i.e. that we could write:

sched = {'lr': [SchedCos(lr/10.,lr) for lr in lrs]}

learner.fit(1, cbs=ParamScheduler(sched))

that is not allowed at the moment.

Why the ActivationStats Callback just records statistics from the last layer of the AWD_LSTM default Language Model? When I type :

learner.activation_stats.stats[0]

gives me back:

[None,None,None,None,None,None,None,None,{'mean': -1.8462823629379272, 'std': 1.2596983909606934, 'near_zero': 0.9194439246831189, 'hist': tensor([1.2411e+06, 1.0021e+06, 8.0230e+05, 6.4066e+05, 5.0985e+05, 4.0494e+05,
        3.2018e+05, 2.5357e+05, 2.0129e+05, 1.5805e+05, 1.2631e+05, 1.0001e+05,
        8.0982e+04, 6.4612e+04, 5.2268e+04, 4.2857e+04, 3.4609e+04, 2.8632e+04,
        2.3751e+04, 1.9741e+04, 1.6459e+04, 1.3934e+04, 1.1901e+04, 1.0163e+04,
        8.5460e+03, 7.2550e+03, 6.3900e+03, 5.4860e+03, 4.8030e+03, 4.0000e+03,
        3.4060e+03, 2.9320e+03, 2.6210e+03, 2.2900e+03, 1.9420e+03, 1.6760e+03,
        1.4160e+03, 1.2110e+03, 1.0560e+03, 8.4500e+02])}]

So it is just tracking the statistics of the 9th layer.

Thanks a lot!!!

marcossantana · October 6, 2020, 2:49am

Any updates on this?

marcossantana · October 22, 2020, 2:53am

Hey @vrodriguezf
did you manage to solve it? I’m facing the same problem.