Metadata of a Tensor Subclass not available in `show_batch`

I’m having trouble to access metadata of a Tensor subclass in show_batch():

class T(TensorBase):
    @classmethod
    def create(cls, data, metadata):
        return cls(data, metadata=metadata)

@typedispatch
def show_batch( x: T, y, samples, ctxs=None, figsize=None, **kwargs):
    print("Metadata:", samples[0][0].metadata)

t = T.create([1,2,3], metadata=1)
assert t.metadata == 1

dls = DataLoaders.from_dsets([t, t], [t, t], bs=2)
dls.show_batch()

Gives an AttributeError: 'T' object has no attribute 'metadata' in my defined show_batch.

Previous to torch 1.9.0, list(b) would return a list of Tensors, not Ts. That’s why retain_types behaves differently and does not copy metadata. If I patch batch_to_samples, so that it does not use list(b) but [b[i] for i in range(..), it seems to work:

def batch_to_samples(b, max_n=10):
    "'Transposes' a batch to (at most `max_n`) samples"
    if isinstance(b, Tensor):
        return retain_types([b[i] for i in range(min(max_n, len(b)))], [b])
    else:
        res = L(b).map(partial(batch_to_samples,max_n=max_n))
        return retain_types(res.zip(), [b])

@patch_to(TfmdDL)
def _decode_batch(self, b, max_n=9, full=True):
    f = self.after_item.decode
    f1 = self.before_batch.decode
    f = compose(f1, f, partial(getattr(self.dataset,'decode',noop), full = full))
    return L(batch_to_samples(b, max_n=max_n)).map(f)
    
dls = DataLoaders.from_dsets([t, t], [t, t], bs=2)
dls.show_batch()

Output:

Metadata: 1

1 Like

Hey @d2weber,

I am having exactly the same problem, and your patches fix it completely. Thanks you very much for sharing.

I have been looking for the last couple of hours into your solution and the offending code in the code-base, but so far it’s still totally beyond my mind how you came up with this fix.

I understand that this post is really quite old, and I don’t think you are active anymore on this forum.

But if so, and you are willing to share how you debugged this, please let me know. I think I can learn a lot from that process.

Thanks again

Hi,
I don’t remember how I debugged this. I think I was wondering what happened in show_batch, so I looked at all the functions getting called in fastai/core.py at 74fd6ea8cca582cc878d89658541873761ecb617 · fastai/fastai · GitHub.
Maybe I copied some of the functions in a ipynb cell and checked when the metadata was gone, until I finally found the culprit. But honestly I don’t understand exactly how all this retain_type stuff really works and if the behaviour we encountered is intentional.
Hope that makes some sense to you, let me know if there is something specific that I can explain.

Hi @d2weber,

Thanks so much for your reply. I also have a difficult time wrapping my head about all these retain_type functionalities. I think it has to do with collating individual items into a batch, and somehow keeping the metadata on that new collated object.

For my project I am trying to circumvent all this stuff now, by trying to not use any metadata at all…

In the process, I wondered whether it would make more sense to switch to raw PyTorch instead of trying to hack my way through the fastai library. The idea starts to emerge that fastai is great for quick “off the shelf solutions” but tricky to customize to non-standard problems.

Since you are probably much further in your “deep learning career” (since you wrote entry about a year ago), what’s your take on that? Do you still use fastai?

Would be very helpful to have your (opinionated) ideas on this :pray:

Even though I worked with the fastai library every now and then within the last couple of month, I don’t consider myself an expert of the deeper fastai functionalities.

My take on that is, that in the beginning it’s very confusing, because so many concepts of fastai (and fastcore) are implicit. So once you start to understand the concepts it gets better. But in the end to me it feels wrong to take these kinds of shortcuts and I would prefer a more explicit API. The first example that comes to my mind is the DataBlock API, it’s implicit that if you set n_inp, all the other blocks in are output blocks. So if someone reads the code, he has to know these conventions (and that part is not even documented here.) I don’t understand why the input and output blocks are passed as a single list in the first place. And similar to this there are many conventions and shortcuts that you have to learn.

That being said, I think I would still prefer using the fastai dataloading functionality over plain pytorch, because I believe fastai is the most advanced and most comfortable to use library (haven’t checked for alternatives lately though). So, depending on your task, it might be worth the effort to learn the fastai concepts and benefit from the great performance and good Integration with the rest of fastai.

Actually fastai amis to be super flexible by giving you the lower level tools to incorporate your custom task. A good starting point to understand the dataloading part of fastai is always the siamese toutorial.

1 Like