Developer chat

sgugger · February 15, 2019, 2:49pm

Fixed in master.

pierreguillou · February 15, 2019, 7:14pm

Thanks Sylvain. I changed in my local lib the fastai/callback.py file but now I got another error:

AttributeError: 'list' object has no attribute 'parameters'

nandakumar212 · February 15, 2019, 8:20pm

Facing the same issue in colab:
name ‘split_bn_bias’ is not defined

sgugger · February 15, 2019, 10:30pm

I have no idea where this one comes from, I’ll need a reproducible example to fix it.

pierreguillou · February 15, 2019, 11:12pm

(I’m using fastai 1.0.45 on Windows 10) The error appears when I run the lesson7-wgan.ipynb notebook.

nandakumar212 · February 15, 2019, 11:23pm

I am using Google Colab and installed the library using https://course.fast.ai/setup/colab
In the notebook lesson7-superres-gan.ipynb, following error is thrown while calling fit() on GANLearner:

stas · February 16, 2019, 4:59am

I think learn.get_preds() isn’t CLI friendly. it sends \r which impacts user’s console outputs. For example, this test we have:

def test_get_preds():
    learn = fake_learner()
    a = learn.get_preds()
    assert learn.data.batch_size == len(a[1])

was resulting in the test name disappearing (see the first PASSED is lacking its name?)

collected 3 items                                                                                                                                                          

PASSED                                                                                                                                       
tests/test_basic_train.py::test_save_load PASSED
tests/test_basic_train.py::test_save_load_mem_leak PASSED

I had to capture its stdout to fix that:

def test_get_preds():
    learn = fake_learner()
    with CaptureStdout() as cs:
        a = learn.get_preds()
    assert learn.data.batch_size == len(a[1])

now we get:

collected 3 items

tests/test_basic_train.py::test_get_preds PASSED
tests/test_basic_train.py::test_save_load PASSED
tests/test_basic_train.py::test_save_load_mem_leak PASSED

but perhaps it’s ok and we just need to document this side effect and how to overcome it?

sgugger · February 16, 2019, 2:18pm

Ok, I was stupid with my first fix, now it’s really fixed on master.
@nandakumar212 fixed on master means you won’t have the fix in colab until the next release, unless you do a dev install.

pierreguillou · February 16, 2019, 5:58pm

Sylvain, many thanks. The lesson7-wgan.ipynb notebook works well now in master with your change in fastai/callback.py.

vha14 · February 17, 2019, 7:47pm

I was running into issue while experimenting with object detection models. None of the methods Learner.predict, Learner.pred_batch, Learner.get_preds works. The one method that works is Learner.show_results. I saw a TODO by Jeremy back in December about refactoring the code there to work with pred_batch(reconstruct=True). The use of attaching RecordOnCPU callback to capture the input and target is also rather unintuitive to follow. Is there any new thinking/progress on this? (I have to admit, the library’s intricate design is powerful but also quite formidable for a newcomer to grok. Awesome work and kudos nevertheless!)

sgugger · February 18, 2019, 2:18pm

Yes, making object detection work end to end is on our TODO and will be done before the second part of the course begins, but it probably won’t fully work right now. I’d stick with calling the models explicitly for now, until we have sorted this out.

andrijdavid · February 19, 2019, 7:23am

I have written a MultiTask API for one of my projects. It extends the DataBlock Api. I don’t know if it worths to add it to the library.

gist.github.com

https://gist.github.com/andrijdavid/9930154d649f2174dd712a6708b9ec32

data.py

from fastai.basics import ItemBase, ItemList, Iterator, Collection, Callable, LabelList, Any, IntsOrStrs, listify, show_some, array, extract_kwargs
from fastai.vision import ImageDataBunch, ImageItemList
from fastai.data_block import PreProcessors, PreProcessor

class MultiLabelProcessor(PreProcessor):
    def __init__(self, ds:Collection=None):
        self.ref_ds = ds
        self.procs_y = []

    def process(self, ds:Collection=None):

This file has been truncated. show original

learner.py

from fastai.vision.learner import cnn_config
from fastai.vision import *
from fastai.callbacks.hooks import num_features_model

class MultiHeadWrapper(nn.Module):
    def __init__(self, head):
        super().__init__()
        self.pool = nn.Sequential(AdaptiveConcatPool2d(), Flatten())
        self.head = nn.ModuleList()
        for h in head:

This file has been truncated. show original

sgugger · February 19, 2019, 2:12pm

That looks very nice! It may be a bit specific to be included in fastai, but we can definitely link to it with other examples of custom ItemList or custom model definitions.

sgugger · February 20, 2019, 10:58pm

Finally, text in mixed precision is debugged and I can properly train an AWD-LSTM in mixed precision. QRNNs don’t support mixed-precision training however.
Haven’t tried Transformer and TransformerWL but there is no reason it shouldn’t work.

Benudek · February 21, 2019, 7:19am

@sgugger will you record that, share slides? https://twitter.com/GuggerSylvain/status/1097893976072425472

Actually, it was an area that looked interesting and complex, so would be cool. Maybe even a summary in the docs at some point based on what you share

stas · February 21, 2019, 8:19pm

Please have a look at:

and give us feedback. This just came out.

sgugger · February 21, 2019, 8:50pm

I’ll share what I can, yes.

Kaspar · February 22, 2019, 2:06pm

would be really usefull for tranformerxl - it really used a lot of gpu mem

Carl1 · February 24, 2019, 7:23am

hi, @deena-b i get the same issue, have you solved it ?

Kaspar · February 24, 2019, 7:35am

looks like tranformerXL is cpu bound - my cpu utilization is higher than the gpu utilization. Is that also your experience @sgugger