Multiple Pipelines DataBlock, making show methods work

fastai fastai dev
1

Hello everyone,
I am looking on info on how to build a DataBlock with multiple pipelines at the same time. I have succesufully build a Block with 2 inputs and 1 target doing this: (thanks @muellerzr)

block=DataBlock(blocks = (ImageTupleBlock, TSBlock, TSBlock),
              getters = [lambda t: t[0][0], lambda t: t[0][1], lambda t: t[1]],
              splitter = IndexSplitter(val_idxs),
              n_inp    = 2)

but I would like to make show_batch and show_results work. The output I had using a custom class to wrap together ImageTuple+TS was this:

image
image812Ă—512 73.4 KB

Here a ImageTuple is shown at left and the timeseries at right. Stacking together the x timeseries and the y, similar to @takotab fastseq package.
This was done using a typed version of the show_batch method.

@typedispatch
def show_batch(x: ImageFeatureTuple, y: TSeries, samples, ctxs=None, max_n=10, nrows=None, figsize=None, fsteps=0, **kwargs):
    "Show batch for TSeries objects"
    if ctxs is None: _, ctxs = plt.subplots(min(len(samples), max_n) ,2, figsize=(10,3*len(samples)))
    for b, tar, i in zip(samples.itemgot(0), samples.itemgot(1), range(max_n)):
        b.show(ctx=ctxs[i], label='x (ghi)', bkgd=True)
        tx=np.arange(fsteps, b[-1].shape[1]+fsteps)
        tar.show(ctx=ctxs[i][1], label='y (ghi)', color='--*r', tx=tx, **kwargs)
    return ctxs

I want a more elegant solution that is compatible with the rest of fastai and that works for my new 3 pipelines block. Probably I will need the bricolaging power of @sgugger here

2 Likes
2

There is no way to use three different types in show_batch: it’s been designed for two only. So using the typed tuple approach is the best we can do.

1 Like
3

How does it work then in the case of Bounding Boxes?

4

I suppose we work with same axes in case of bounding boxes, so a general

def show_batch(x:TensorImage, y, samples, ...):

do the job

5

Would you like to have ImageSequence support on fastai?

1 Like
6

I may need to keep my ugly ImageFeatureTuple class then…

7

Absolutely!