Fastai v2 vision

You can also just pass the function PILMask.create as an item_tfms (in DataBlock) or after_item (in DataSource),

1 Like

I’m trying Kaggle’s Understanding Clouds competition, and would like to have , in the modeling data, the image as the independent variable and four masks, one for each cloud type, as the dependent variables.

Right now, I’m passing a tfms containing 5 transform pipelines to DataSource. The first one for the image, and the other four for the masks.

tfms = [[PILImage.create], 
        [RLE_Decode('fish', annots, IMG_SHAPE), PILMask.create],
        [RLE_Decode('flower', annots, IMG_SHAPE), PILMask.create],
        [RLE_Decode('gravel', annots, IMG_SHAPE), PILMask.create],
        [RLE_Decode('sugar', annots, IMG_SHAPE), PILMask.create]]

dsrc = DataSource(items, tfms=tfms)

and from this, the DataBunch is created, with:

after_item_tfms = [ToTensor,]
after_batch_tfms = [Cuda(), IntToFloatTensor()]
dbch = dsrc.databunch(after_item=after_item_tfms, after_batch=after_batch_tfms, bs=4)

which seems to load things of the desired shapes:

So, the next step will be to pass the DataBunch to Learner along with the model, loss, etc. However, in this whole process I haven’t told Fastai which of the tuple of 5 things are the independent variables and which are the dependent variables. How will it know to treat just the first tensor as the input to the model, and the rest as the targets?

My notebook for this can be found here.

There is a keyword argument called n_inp for DataSource. This is probably the number of independent variables. DataBlock also has it, too.

Yes, that’s exactly it. Pass along n_inp=1 because by default, the library considers everything but the last tensor as the input. Note that your loss function will need to take one output (which will very likely be a list of 4 things or a huge tensor that you will split in 4) and 4 targets.

1 Like

When I use cnn_learner with [accuracy,RocAuc] or [accuracy,RocAucMulti] metrics I got and error on windows. It it wrong use ? How can I fix it ?

cnn_learner(den_dbn, xresnet50, opt_func=opt_func, metrics=[accuracy,RocAuc])

some more error mesages

d:\codes\fastai_dev\dev\local\ in accumulate(self, learn)
399 def accumulate(self, learn):
400 bs = find_bs(learn.yb)
–> 401 += to_detach(_maybe_reduce(self.func(learn.pred, *learn.yb)))*bs
402 self.count += bs
403 @property

TypeError: unsupported operand type(s) for *: ‘AccumMetric’ and ‘int’

You need to instantiate your RocAuc metric: pass RocAuc()

1 Like

thank you for your support. After correcting it I recive

d:\codes\fastai_dev\dev\local\ in _maybe_item(t)
435 def _maybe_item(t):
–> 436 t = t.value
437 return t.item() if isinstance(t, Tensor) and t.numel()==1 else t

d:\codes\fastai_dev\dev\local\ in value(self)
41 preds,targs =,
42 if self.to_np: preds,targs = preds.numpy(),targs.numpy()
—> 43 return self.func(targs, preds, **self.kwargs) if self.invert_args else self.func(preds, targs, **self.kwargs)
45 #Cell

TypeError: recall_score() got an unexpected keyword argument ‘laverage’

Oh this one is a typo. Fixing now.

ok thank you

@sgugger, sorry to bother again … but this time I get err:

d:\codes\fastai_dev\dev\local\ in value(self)
42 preds,targs =,
43 if self.to_np: preds,targs = preds.numpy(),targs.numpy()
—> 44 return self.func(targs, preds, **self.kwargs) if self.invert_args else self.func(preds, targs, **self.kwargs)
46 #Cell

TypeError: recall_score() got an unexpected keyword argument ‘max_fpr’

Ah yes, copy-pasting error. The function called was recall_score and not auc_roc_score. Fixed now.

I’m attempting to do segmentation right now, I have the model trained and I’m trying to call show_results like so (assume up until this point I have been following CAMVID):

learn.show_results(max_n=4, vmin=1, vmax=30, figsize=(15,6))

local variable 'old_shuffle' referenced before assignment

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/fastai2/ in _do_epoch_validate(self, ds_idx, dl)
    254         try:
--> 255             dl.shuffle,old_shuffle = False,dl.shuffle
    256             dl.drop_last,old_drop = False,dl.drop_last

AttributeError: 'list' object has no attribute 'shuffle'

Along with this:

<ipython-input-51-67d4997d7834> in <module>()
----> 1 learn.show_results(max_n=4, figsize=(15,6))

/usr/local/lib/python3.6/dist-packages/fastai2/ in show_results(self, ds_idx, dl, max_n, **kwargs)
    318         if dl is None: dl = self.dbunch.dls[ds_idx]
    319         b = dl.one_batch()
--> 320         _,_,preds = self.get_preds(dl=[b], with_decoded=True)
    321         self.dbunch.show_results(b, preds, max_n=max_n, **kwargs)

/usr/local/lib/python3.6/dist-packages/fastai2/ in get_preds(self, ds_idx, dl, with_input, with_loss, with_decoded, act, save_preds, save_targs)
    294         with self.no_logging(), self.added_cbs(cb), self.loss_not_reduced(), self.no_mbar():
    295             self(_before_epoch)
--> 296             self._do_epoch_validate(ds_idx, dl)
    297             self(_after_epoch)
    298             if act is None: act = getattr(self.loss_func, 'activation', noop)

/usr/local/lib/python3.6/dist-packages/fastai2/ in _do_epoch_validate(self, ds_idx, dl)
    259         except CancelValidException:                         self('after_cancel_validate')
    260         finally:
--> 261             dl.shuffle,dl.drop_last = old_shuffle,old_drop;  self('after_validate')
    263     def fit(self, n_epoch, lr=None, wd=defaults.wd, cbs=None, reset_opt=False):

UnboundLocalError: local variable 'old_shuffle' referenced before assignment

Perhaps it’s grabbing both of the DataLoaders instead of one?

I thing It’s broken recently last week it was working… I had the same issue with classification with 4 classes of images…

Yes Jeremy broke this during the weekend while adding some other functionality. Will fix this morning.

Edit: Should be fixed now.

1 Like

So I have a question on the DataBlock API. I’m going through object detection at the moment and I notice we have three blocks here: ImageBlock, BBoxBlock, BBoxLblBlock. My understanding of the blocks was that we should make a tuple around what’s our input blocks and our output blocks, and that block wants a tuple of 2. Eg: blocks = ((ImageBlock), (BBoxBlock, BBoxLblBlock))

Do we just in general say what overall blocks we will be using? Or is that when we declare n_inp=1 so it will assume the output is 2. (the other two blocks)

Last question, why is noop necessary? (check me if I’m wrong here) but can we assume we should have a getter place for each Block?

Code I am talking about:

coco = DataBlock(blocks=(ImageBlock, BBoxBlock, BBoxLblBlock),
                 getters=[noop, lambda o: img2bbox[][0], lambda o: img2bbox[][1]], n_inp=1)

Also any thought to adding RetinaNet into the library?

My guess is that the way to go is the latter, using (ImageBlock, BBoxBlock, BBoxLblBlock) with n_inp. This ensures that at any stage in the data processing pipeline, there is a Tuple that is just one level deep.

((ImageBlock), (BBoxBlock, BBoxLblBlock)) is a nested tuple, and I don’t think Transform acts on elements that are inside a nested tuple. So, if you try to apply a rotation transform, it will just rotate ImageBlock, but not BBoxBlock or BBoxLblBlock.

1 Like

Makes sense. The latter was in the DataBlock notebook, just trying to understand how we determine inputs and outputs. So it’s safe to assume it will search for the first x blocks to make our inputs those types? And the rest will be outputs?

I’ve tried using n_inp=1 with DataSource for a tuple of length 5 (so 1 input + 4 outputs) and everything seems to work, up until, and including, calling

I’ve sort of managed to train such a model, but am running into an error when using Learner's predict and show_results methods. For example, when using show_results:

TypeError                                 Traceback (most recent call last)
<ipython-input-91-9ca34d9564c5> in <module>
----> 1 learn.show_results(max_n=4)

~/fastai_dev/dev/local/ in show_results(self, ds_idx, dl, max_n, **kwargs)
    316         if dl is None: dl = self.dbunch.dls[ds_idx]
    317         b = dl.one_batch()
--> 318         _,_,preds = self.get_preds(dl=[b], with_decoded=True)
    319         self.dbunch.show_results(b, preds, max_n=max_n, **kwargs)

~/fastai_dev/dev/local/ in get_preds(self, ds_idx, dl, with_input, with_loss, with_decoded, act, save_preds, save_targs)
    297             res = []
    298             if len(cb.preds):
--> 299                 preds = act(
    300                 res.append(preds)
    301                 if with_decoded: res.append(getattr(self.loss_func, 'decodes', noop)(preds))

TypeError: expected Tensor as element 0 in argument 0, but got tuple

As far as I can tell, it’s because the model’s output is a tuple of tensors, instead of just a tensor. The exact dimensions of the model output and targets in this instance are:


The images and masks are 210 px in width and 140 px in height. The batch size is 9. The target is a tuple of 4 masks, and the model outputs a 2x140x210 tensor for each of these masks.

Are there some ways that get_preds can be generalized to handle such situations?

Is the in preds = act( to concatenate things from different batches?

What is the current formal way of obtaining predictions on a test set? I see from this Kaggle starter notebook from @radek that, to do this, a separate DataBunch with dummy targets is created, then it is assigned as Learner.dbunch, and then finally Learner.get_preds is called to get the predictions, along with the targets.