Fastai v2 vision

This gives me:

TypeError                                 Traceback (most recent call last)
<ipython-input-20-4aa5133ccf6a> in <module>
      1 batch = dls.one_batch()[0]
----> 2 res = learn.model(batch)

/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    530             result = self._slow_forward(*input, **kwargs)
    531         else:
--> 532             result = self.forward(*input, **kwargs)
    533         for hook in self._forward_hooks.values():
    534             hook_result = hook(self, input, result)

TypeError: forward() missing 1 required positional argument: 'x2'

len(batch) = 3

D’oh. Just realized you did the above in your post. My bad!!!

No problem. What I’m doing here mainly is that I’m replacing SequentialEx with CustomSequentialEx (that can handle 2 inputs and ignores the second input for the moment). The rest is very close to CamVid.
Only that the dataloaders has 2 inputs.

So again this probably has to do with the y’s (this is when I saw these errors popping up). We discussed this in the study group (but it’s buried). Try incorporating the ideas from this thread to make sure you didn’t miss any classes:

I’m checking this right away but that is strange because I’m using almost the same DataBlock as the one from https://github.com/fastai/fastai2/blob/master/nbs/examples/camvid.ipynb. Except:

camvid = DataBlock(blocks=(ImageBlock, MaskBlock(codes)) =>
camvid = DataBlock(blocks=(ImageBlock, MaskBlock(codes), MaskBlock(codes)),

and n_inp=2

And you’re using the camvid dataset?

Yes, for the moment I’m only trying to implement this 2 inputs DynamicUnet. And I’m testing it on CamVid.
Only after I’ll plug my real datasets.

Hmm. Unsure. @sgugger May be able to chime in better :slight_smile:

It seems to be related to some sort of serialization:

(this comes from the previous error trace)

~/workspace/fastai2/fastai2/callback/schedule.py in after_fit(self)
    168         tmp_f = self.path/self.model_dir/'_tmp.pth'
    169         if tmp_f.exists():
--> 170             self.learn.load('_tmp')
    171             os.remove(tmp_f)
    172

The problem is caused when the predictions, in a mutli-label problem, contain no categories that are predicted to be true. So in the attached image you can see that all of the predictions for the first image in the batch are set to False.

In the current implementation of the show_results function there is no check to see if at least one prediction exists, and so it causes a crash.

In my modified function it checks if “r.show” exists before using it and this therefore stops the crash & prints the target label in black.

Just wanting to point out a bug, I’m unsure how exactly we could fix it without type dispatch, but considering it’s in the batch level we lost this ability. I’m attempting to do Mask RCNN and so I’m building a DataBunch that contains bounding boxes and segmentation masks. Everything works fine up until bb_pad, and this is due to it expecting in the batch, samples 1 and on to contain the bbox and labels and that is it.

My fix was:

    if len(samples[0]) > 3:
      samples = [(s[0], *clip_remove_empty(*s[1:3])) for s in samples]
    else:
      samples = [(s[0], *clip_remove_empty(*s[1:])) for s in samples]

However this isn’t perfect as we’re forced to make the user assume to pass in the bounding boxes first and then any other datatypes we want. Any ideas? (@sgugger)

bb_pad_collate doesn’t type-dispatch and is only working with object detection, you should write your own version for other problems.

1 Like

Got it, thanks :slight_smile: for anyone wanting to do something similar, s reads in your samples, so this (in my case) was set as the following:

input image, bbox coord, bbox lbl, segmentation mask.
So I cared about 1:3 for the bounding box paddings

I then just did myBBoxBlock = TransformBlock(type_tfms=TensorBBox.create, item_tfms=PointScaler, dls_kwargs = {'before_batch': mybb_pad}) :slight_smile:

2 Likes

I was trying to build my SemanticTensor pipeline but it throws error if use my class instead of normal tensor:

class TensorContinuous(TensorBase): pass

class NormalizeTfm(Transform):
  def encodes(self, o): return (o - self.mean) / self.std
  def decodes(self, o): return (o * self.std) + self.mean
  def setups(self, dsets):
    td = tensor(dsets).float()
    self.mean, self.std = td.mean(), td.std()+1e-7

class RegSetup(Transform):
    "Transform that floatifies targets"
    def encodes(self, o): return tensor(o).float()
    # def encodes(self, o): return TensorContinuous(o).float()
    def decodes(self, o):return TitledStr(o.item())

def RegressionFBlock():
  return TransformBlock(type_tfms=[RegSetup(), NormalizeTfm()])

dblock = DataBlock(blocks=(ImageBlock, RegressionFBlock, RegressionFBlock, CategoryBlock),
                   getters=getters,
                   splitter=ColSplitter('is_val'),
                   item_tfms=Resize(size),
                   batch_tfms = [*aug_transforms(max_zoom=0, flip_vert=True)])

RegSetup was originally RegressionSetup. I just cleared out some unnecessary code for me.
Above code works and I do get normalized column tensor. However, if I change the encodes method of RegSetupto commented one, it throws following error:

RuntimeError: detach is not implemented for Tensor

Also, let me know if I could move the NormalizeTfm to batch_tfms somehow.

Thanks!

Trying to work my way with the SiamesePair example right now, and I’m confused with one question. So where do we apply this? I’m having issues connecting the dots together from our Pipeline into the DataSource/DataLoaders. Where would we load in the SiamesePair transform? (I’ve tried the obvious bits IE after_ and inside the tfms of our Datasets transforms. The last bit I may do is make a custom transform a little differently)

cc @sgugger?

No, the transform is applied in a TfmdLists that you then convert to a DataLoaders with the .dataloaders method.

Thanks, I think I understand now :slight_smile: So I could do the following then?

tl = TfmdLists(items[:10], pipe)
dl = tl.dataloaders(bs=1)

If so I get a PosixPath object is not iterable error. (this is just passing in a list of items.) If I try passing in instead a list of x’s and y’s from a dset I get “TypeError: int() argument must be a string, a bytes-like object or a number, not ‘PILImage’”

(said code from tutorial and how I set it up):

class SiamesePair(Transform):
    def __init__(self,items,labels):
        self.items,self.labels,self.assoc = items,labels,self
        sortlbl = sorted(enumerate(labels), key=itemgetter(1))
        # dict of (each unique label) -- (list of indices with that label)
        self.clsmap = {k:L(v).itemgot(0) for k,v in itertools.groupby(sortlbl, key=itemgetter(1))}
        self.idxs = range_of(self.items)
        
    def encodes(self,i):
        "x: tuple of `i`th image and a random image from same or different class; y: True if same class"
        othercls = self.clsmap[self.labels[i]] if random.random()>0.5 else self.idxs
        otherit = random.choice(othercls)
        return SiameseImage(self.items[i], self.items[otherit], self.labels[otherit]==self.labels[i])

class SiameseImage(Tuple):
    def show(self, ctx=None, **kwargs): 
        img1,img2,same_breed = self
        return show_image(torch.cat([img1,img2], dim=2), title=same_breed, ctx=ctx)

OpenAndResize = Transform(resized_image)
labeller = RegexLabeller(pat = r'/([^/]+)_\d+.jpg$')
sp = SiamesePair(items, items.map(labeller))
pipe = Pipeline([sp, OpenAndResize])

dset = pets.datasets(src)
tl = TfmdLists(dset[:10], pipe)
dl = tl.dataloaders(bs=1)

This is because i at this point is an image itself and a label

Sorry just the answer isn’t obvious to me on how to go from A->B here :sweat_smile:

1 Like

Your transform to resize should be on as an after_item in your dataloaders, it will automatically be applied to every part of the tuple that needs it.

I’m still having the PosixPath issue. Which specifically happens here:
othercls = self.clsmap[self.labels[i]] if random.random()>0.5 else self.idxs

The issue itself is in the SiamesePair transform, I guess I’m confused with how it expects the data to be loaded in outside of a Pipeline. I think we still need to make an instance of it yes? (Or no because it should be flexible depending on the input.) If not, how do we pass in it’s items and labels we want.

(I guess it may have been meant to be as a show example and should be refactored if actually applying to a DataLoader?)

@sgugger please have a look.

Thanks!