Developer chat

sgugger · February 18, 2019, 2:18pm

Yes, making object detection work end to end is on our TODO and will be done before the second part of the course begins, but it probably won’t fully work right now. I’d stick with calling the models explicitly for now, until we have sorted this out.

andrijdavid · February 19, 2019, 7:23am

I have written a MultiTask API for one of my projects. It extends the DataBlock Api. I don’t know if it worths to add it to the library.

gist.github.com

https://gist.github.com/andrijdavid/9930154d649f2174dd712a6708b9ec32

data.py

from fastai.basics import ItemBase, ItemList, Iterator, Collection, Callable, LabelList, Any, IntsOrStrs, listify, show_some, array, extract_kwargs
from fastai.vision import ImageDataBunch, ImageItemList
from fastai.data_block import PreProcessors, PreProcessor

class MultiLabelProcessor(PreProcessor):
    def __init__(self, ds:Collection=None):
        self.ref_ds = ds
        self.procs_y = []

    def process(self, ds:Collection=None):

This file has been truncated. show original

learner.py

from fastai.vision.learner import cnn_config
from fastai.vision import *
from fastai.callbacks.hooks import num_features_model

class MultiHeadWrapper(nn.Module):
    def __init__(self, head):
        super().__init__()
        self.pool = nn.Sequential(AdaptiveConcatPool2d(), Flatten())
        self.head = nn.ModuleList()
        for h in head:

This file has been truncated. show original

sgugger · February 19, 2019, 2:12pm

That looks very nice! It may be a bit specific to be included in fastai, but we can definitely link to it with other examples of custom ItemList or custom model definitions.

sgugger · February 20, 2019, 10:58pm

Finally, text in mixed precision is debugged and I can properly train an AWD-LSTM in mixed precision. QRNNs don’t support mixed-precision training however.
Haven’t tried Transformer and TransformerWL but there is no reason it shouldn’t work.

Benudek · February 21, 2019, 7:19am

@sgugger will you record that, share slides? https://twitter.com/GuggerSylvain/status/1097893976072425472

Actually, it was an area that looked interesting and complex, so would be cool. Maybe even a summary in the docs at some point based on what you share

stas · February 21, 2019, 8:19pm

Please have a look at:

and give us feedback. This just came out.

sgugger · February 21, 2019, 8:50pm

I’ll share what I can, yes.

Kaspar · February 22, 2019, 2:06pm

would be really usefull for tranformerxl - it really used a lot of gpu mem

Carl1 · February 24, 2019, 7:23am

hi, @deena-b i get the same issue, have you solved it ?

Kaspar · February 24, 2019, 7:35am

looks like tranformerXL is cpu bound - my cpu utilization is higher than the gpu utilization. Is that also your experience @sgugger

stas · February 24, 2019, 5:43pm

Some handy pytest extensions I have just discovered (click on the links for the full description, installation info).

Automatically rerun failed tests on source modification
```
pytest -f
```
Run only modified tests
```
pytest --picked
```
improved look-n-feel: add progress-bar and show failing tests immediately

I’ve been using this last plugin today a lot in various projects and I love it!

deena-b · February 24, 2019, 6:05pm

Hi @Carl1, I haven’t updated pytorch and fastai yet, but it sounded like that would fix it.

stas · February 25, 2019, 9:35pm

2 posts were split to a new topic: Shortest failing test sequence finder (coupled tests detector)

sgugger · February 25, 2019, 8:50pm

Breaking change

Did you all notice how we had ImageItemList but TextList and TabularList? Well not anymore. In master and v1.0.46 ImageItemList becomes ImageList.

stas · February 25, 2019, 9:37pm

I could use some CS algorithm help here.

Needing help with a search algorithm to find the shortest sequence of items that lead to a certain outcome (failed test in this case). Please see this post.

Thank you.

stas · February 25, 2019, 11:01pm

@sgugger has just released fastai-1.0.46, list of changes.

Also there was a bit of a mess up with 1.0.44 vs 1.0.45, which are really the same release. So I removed 1.0.45 from pypi/conda and there is 1.0.44 instead for those who want it. But, of course, 1.0.46 is out.

imaginary · March 1, 2019, 2:47pm

ImageItemList was renamed to ImageList for consistency and at the same time MixedItemList was introduced?

Looking forward to try the new TextClassificationInterpretation function!

sgugger · March 1, 2019, 2:58pm

That is correct. MixedItemList is a mix of ItemLists (and experimental for now), so there is no inconsistency here.

amqdn · March 1, 2019, 11:26pm

Hey, Jeremy –

Looks like the keyboard shortcut to bring up the list of Discourse shortcuts has been changed to shift-/ instead of just /, at least on Chrome on Linux. If this is universal, could you please edit your initial post to reflect this? Thanks!

stas · March 2, 2019, 2:56am

Edited. Thank you for the heads up, @amqdn