Solving "kwargs" issue inside the "ImageBlock" class - help needed

Danrohn · July 16, 2022, 12:06am

Hey folks,
I hope that it’s okay to publish it here as a separate topic, but if not, maybe one moderator would redirect it.

So I need some help on a technical issue, which is probably solvable by some Python tweaks.

I’ve created a new class RawImage, like PILImage, and passed it in here:

# Cell
def ImageBlock(cls=RawImage):
    "A `TransformBlock` for images of `cls`"
    return TransformBlock(type_tfms=cls.create, batch_tfms=IntToFloatTensor)

Now I want to pass in **kwargs into RawImage, which would be later on passed into cls.create.

Something like this:

# Cell
def ImageBlock(cls=RawImage(**kwargs)):
    "A `TransformBlock` for images of `cls`"
    return TransformBlock(type_tfms=cls.create(**kwargs), batch_tfms=IntToFloatTensor)

or:

# Cell
def ImageBlock(cls=RawImage,**kwargs):
    "A `TransformBlock` for images of `cls`"
    return TransformBlock(type_tfms=cls.create(**kwargs), batch_tfms=IntToFloatTensor)

But I get this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)

<ipython-input-64-815c5e98f4ac> in ImageBlock(cls, **kwargs)
     75 def ImageBlock(cls=RawImage,**kwargs):
     76     "A `TransformBlock` for images of `cls`"
---> 77     return TransformBlock(type_tfms=cls.create(**kwargs), batch_tfms=IntToFloatTensor)
     78 
     79 # Cell

TypeError: create() missing 1 required positional argument: 'fn'

This is my RawImage’s create of the class:

class RawImage(A):
    "for fast.ai"
    @classmethod
    def create(cls,fn:Path, **kwargs)->None:
        return cls(fn,**kwargs)

I’m not sure why I need to pass in fn, because with PILImage it doesn’t ask for fn whenever I pass kwargs as well.

So how can I solve this? What did I do wrong with the concept of kwargs on here?

I don’t get it why here it doesn’t ask for fn:

# Cell
def ImageBlock(cls:PILBase=PILImage):
    "A `TransformBlock` for images of `cls`"
    return TransformBlock(type_tfms=cls.create, batch_tfms=IntToFloatTensor)

Here’s PILImage class:

class PILBase(Image.Image, metaclass=BypassNewMeta):
    _bypass_type=Image.Image
    _show_args = {'cmap':'viridis'}
    _open_args = {'mode': 'RGB'}
    @classmethod
    def create(cls, fn:Path|str|Tensor|ndarray|bytes, **kwargs)->None:
        "Open an `Image` from path `fn`"
        if isinstance(fn,TensorImage): fn = fn.permute(1,2,0).type(torch.uint8)
        if isinstance(fn, TensorMask): fn = fn.type(torch.uint8)
        if isinstance(fn,Tensor): fn = fn.numpy()
        if isinstance(fn,ndarray): return cls(Image.fromarray(fn))
        if isinstance(fn,bytes): fn = io.BytesIO(fn)
        return cls(load_image(fn, **merge(cls._open_args, kwargs)))

Thanks

bencoman · July 16, 2022, 9:19am

Notice in the last line of your examples, when kwargs is dereferenced the double-star is absent…

The same can be seen here, comparing Line 113 and Line 120…

github.com

fastai/fastai/blob/85858cadddd33d9c1ad9b2ed111570dd15bba6bb/fastai/vision/core.py#L113-L120


      
          def create(cls, fn:(Path,str,Tensor,ndarray,bytes), **kwargs)->None:
              "Open an `Image` from path `fn`"
              if isinstance(fn,TensorImage): fn = fn.permute(1,2,0).type(torch.uint8)
              if isinstance(fn, TensorMask): fn = fn.type(torch.uint8)
              if isinstance(fn,Tensor): fn = fn.numpy()
              if isinstance(fn,ndarray): return cls(Image.fromarray(fn))
              if isinstance(fn,bytes): fn = io.BytesIO(fn)
              return cls(load_image(fn, **merge(cls._open_args, kwargs)))

So I’m guessing the following should work…

class RawImage(A):
    "for fast.ai"
    @classmethod
    def create(cls,fn:Path, **kwargs)->None:
        return cls(fn, kwargs)

Danrohn · July 16, 2022, 7:23pm

Hey, thank you for your help!

I had already tried that as well, but it didn’t work out, unfortunately. You could see that the ** was there but before the merge. Anyway, I tried both variations, and it still didn’t work.

It seems to me that passing the cls.create into type_tfms helps later on the Datasets or DataBlock classes to set what Transform there would be used per each item. Basically it passes in the method that is used to open the image file from a filename. Only when passing **kwargs, somewhere in between it realises that it was supposed to get the fn passed in too, but it wasn’t. So it then asserts some error.

I really wish that there was a way to send **kwargs in the ImageBlock(cls=PILImage(**kwargs)) so it can let the user pick what way the PILImage would open the image file (size, quality, etc).

Here’s an error: (I didn’t need to pass in any kwargs, just add the two () in the first instance of PILImage in the ImageBlock.

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-14-4179cfe33bfc> in <module>()
----> 1 dblock = DataBlock(blocks=(ImageBlock(cls=PILImage()), ImageBlock(cls=PILImage)),
      2                    splitter=RandomSplitter(seed=2022),

/usr/local/lib/python3.7/dist-packages/fastcore/meta.py in __call__(cls, x, *args, **kwargs)
     60         if hasattr(cls, '_new_meta'): x = cls._new_meta(x, *args, **kwargs)
     61         elif not isinstance(x,getattr(cls,'_bypass_type',object)) or len(args) or len(kwargs):
---> 62             x = super().__call__(*((x,)+args), **kwargs)
     63         if cls!=x.__class__: x.__class__ = cls
     64         return x

TypeError: __init__() takes 1 positional argument but 2 were given

I tried to fix this by adjusting the ImageBlock, as seen here, but once again another error:

TypeError                                 Traceback (most recent call last)
<ipython-input-9-1dda9d15a3c1> in <module>()
----> 1 dblock = DataBlock(blocks=(ImageBlock(cls=PILImage, check=1), ImageBlock(cls=PILImage)),
      2                    splitter=RandomSplitter(seed=2022),

/usr/local/lib/python3.7/dist-packages/fastai/vision/data.py in ImageBlock(cls, **kwargs)
     84 def ImageBlock(cls:PILBase=PILImage,**kwargs):
     85     "A `TransformBlock` for images of `cls`"
---> 86     return TransformBlock(type_tfms=cls.create(**kwargs), batch_tfms=IntToFloatTensor)
     87 
     88 # Cell

TypeError: create() missing 1 required positional argument: 'fn'

Another update. I tried to add fn=None,**kwargs in the create inside the ImageBlock. Then I got this progress, yet error:

Collecting items from /content/drive/MyDrive/Dataset15/RAW/Short3
Found 107 items
2 datasets of sizes 86,21
Setting up Pipeline: RAWImage
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-6a7e357dcd28> in <module>()
     18                                                 xtra_tfms=RandomErasing(max_count=2, min_aspect=0.3)
     19                                                 )])
---> 20 dls = dblock.dataloaders(source=path_short,bs=btchSize,path=path,verbose=True)
     21 dls.c=3 # Setting n_out // c=Channels

10 frames
/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in dataloaders(self, source, path, verbose, **kwargs)
    156         **kwargs
    157     ) -> DataLoaders:
--> 158         dsets = self.datasets(source, verbose=verbose)
    159         kwargs = {**self.dls_kwargs, **kwargs, 'verbose': verbose}
    160         return dsets.dataloaders(path=path, after_item=self.item_tfms, after_batch=self.batch_tfms, **kwargs)

/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in datasets(self, source, verbose)
    148         splits = (self.splitter or RandomSplitter())(items)
    149         pv(f"{len(splits)} datasets of sizes {','.join([str(len(s)) for s in splits])}", verbose)
--> 150         return Datasets(items, tfms=self._combine_type_tfms(), splits=splits, dl_type=self.dl_type, n_inp=self.n_inp, verbose=verbose)
    151 
    152     def dataloaders(self,

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in __init__(self, items, tfms, tls, n_inp, dl_type, **kwargs)
    452     ):
    453         super().__init__(dl_type=dl_type)
--> 454         self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
    455         self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
    456 

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in <listcomp>(.0)
    452     ):
    453         super().__init__(dl_type=dl_type)
--> 454         self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
    455         self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
    456 

/usr/local/lib/python3.7/dist-packages/fastcore/foundation.py in __call__(cls, x, *args, **kwargs)
     95     def __call__(cls, x=None, *args, **kwargs):
     96         if not args and not kwargs and x is not None and isinstance(x,cls): return x
---> 97         return super().__call__(x, *args, **kwargs)
     98 
     99 # Cell

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in __init__(self, items, tfms, use_list, do_setup, split_idx, train_setup, splits, types, verbose, dl_type)
    366         if do_setup:
    367             pv(f"Setting up {self.tfms}", verbose)
--> 368             self.setup(train_setup=train_setup)
    369 
    370     def _new(self, items, split_idx=None, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in setup(self, train_setup)
    393             for f in self.tfms.fs:
    394                 self.types.append(getattr(f, 'input_types', type(x)))
--> 395                 x = f(x)
    396             self.types.append(type(x))
    397         types = L(t if is_listy(t) else [t] for t in self.types).concat().unique()

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in __call__(self, x, **kwargs)
     71     @property
     72     def name(self): return getattr(self, '_name', _get_name(self))
---> 73     def __call__(self, x, **kwargs): return self._call('encodes', x, **kwargs)
     74     def decode  (self, x, **kwargs): return self._call('decodes', x, **kwargs)
     75     def __repr__(self): return f'{self.name}:\nencodes: {self.encodes}decodes: {self.decodes}'

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _call(self, fn, x, split_idx, **kwargs)
     81     def _call(self, fn, x, split_idx=None, **kwargs):
     82         if split_idx!=self.split_idx and self.split_idx is not None: return x
---> 83         return self._do_call(getattr(self, fn), x, **kwargs)
     84 
     85     def _do_call(self, f, x, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _do_call(self, f, x, **kwargs)
     87             if f is None: return x
     88             ret = f.returns(x) if hasattr(f,'returns') else None
---> 89             return retain_type(f(x, **kwargs), x, ret)
     90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)

/usr/local/lib/python3.7/dist-packages/fastcore/dispatch.py in __call__(self, *args, **kwargs)
    121         elif self.inst is not None: f = MethodType(f, self.inst)
    122         elif self.owner is not None: f = MethodType(f, self.owner)
--> 123         return f(*args, **kwargs)
    124 
    125     def __get__(self, inst, owner):

TypeError: 'RAWImage' object is not callable

bencoman · July 17, 2022, 8:51am

Disclaimer: I’ve only recently been exposed to Python OO (previously had only done single screen sysadmin scripts), so this is a bit of a meandering answer, and I don’t know what is expected knowledge, so hope I’m not telling you how to suck eggs but I noticed… The convention of the cls variable name indicates its expecting a class you are passing an instance, so doubtful that approach has any chance of success.

=============================

And jumping around, not sure if this is useful, but looking at the instance creation of TransformBlock, I see here dls_kwargs …

github.com

fastai/fastai/blob/fa188e5bb7ff7dc6129345a045045e8052427da5/fastai/data/block.py#L18-L30


      
          class TransformBlock():
              "A basic wrapper that links defaults transforms for the data block API"
              def __init__(self,
                  type_tfms:list=None, # One or more `Transform`s
                  item_tfms:list=None, # `ItemTransform`s, applied on an item
                  batch_tfms:list=None, # `Transform`s or `RandTransform`s, applied by batch
                  dl_type:TfmdDL=None, # Task specific `TfmdDL`, defaults to `TfmdDL`
                  dls_kwargs:dict=None, # Additional arguments to be passed to `DataLoaders`
              ):
                  self.type_tfms  =            L(type_tfms)
                  self.item_tfms  = ToTensor + L(item_tfms)
                  self.batch_tfms =            L(batch_tfms)
                  self.dl_type,self.dls_kwargs = dl_type,({} if dls_kwargs is None else dls_kwargs)

I don’t know how dls_kwargs is used deeper in the system, but only found a few examples of its use in this search. Actually, looking at that class definition, it seems it doesn’t subclass and has no methods defined, so it just a data holder – unless I missed something.

=============================

Now here is something that confused me for a bit, so just to make sure its clear with you (it might just be my Smalltalk heritage)… despite the propercase name, ImageBlock is not a class or anything to do with inheritance, its just a plain function, and when its called it returns an instance of TransformBlock…

print( type( ImageBlock ) )
print( type( ImageBlock() ) )
print( type( TransformBlock ) )

<class ‘function’>
<class ‘fastai.data.block.TransformBlock’>
<class ‘type’>

github.com

fastai/fastai/blob/fa188e5bb7ff7dc6129345a045045e8052427da5/fastai/vision/data.py#L84-L86


      
          def ImageBlock(cls:PILBase=PILImage):
              "A `TransformBlock` for images of `cls`"
              return TransformBlock(type_tfms=cls.create, batch_tfms=IntToFloatTensor)

===============================

Here is a log of a bit of experimental exploration I did based off fastbook Chapter 2 to dig deeper into how kwargs is passed through the system. You might gain a bit of insight on what is happening if you take your time looking over the red-highlighted stack traces.

But TL;DR, after getting a bit lost at around the return of MyDataBlock>>datasets()
you can see at the end I tried a completely different approach using partials…
Not sure if that suits your need, but first try experimenting with this…

def blah_create(cls, item, kw1='x', kw2='y'):
    print('\ncls =', cls)
    print('item =', item)
    print('kw1 =', kw1)
    print('kw2 =', kw2)
    
f = partial( blah_create, 1 )
f( 2 )

f = partial( blah_create, kw1='XX' )
f( 1, 2 )

cls = 1
item = 2
kw1 = x
kw2 = y
.
cls = 1
item = 2
kw1 = XX
kw2 = y

Then have a think about whether something like this might help…

class RawImage(PILImage):
    @classmethod
    def create(cls, item, _kw1='x', _kw2='y'):
        print('\ncls =', cls)
        print('item =', item)
        print('kw1 =', _kw1)
        print('kw2 =', _kw2)
 
        instance = PILImage.create(item)
        print('Instance = ', instance)
        return instance
                 
def RawImageBlock(kw1='XX', kw2='YY'):
    f = partial(RawImage.create, _kw1=kw1, _kw2=kw2) 
    return TransformBlock(type_tfms=f, batch_tfms=IntToFloatTensor)
    
bears = DataBlock(
    blocks=(RawImageBlock(kw1='YEAH', kw2='BABY!'), CategoryBlock), 
    get_items=get_image_files, 
    splitter=RandomSplitter(valid_pct=0.2, seed=42),
    get_y=parent_label,
    item_tfms=Resize(128))
dls = bears.dataloaders(path)
dls.valid.show_batch(max_n=4, nrows=1)

jeremy · July 18, 2022, 2:51am

Generally we tend to use such names to indicate things that behave a bit like a constructor, even if it isn’t actually one. It feels like having image_block and TransformBlock would be a bit confusing, and is really indicating a non-essential detail about implementation.

Danrohn · July 20, 2022, 12:58am

Thank you so much for your amazing solution. I also liked your guidance and serving style in the sandbox notebook, as such I like to follow and introduce my thinking process too. Frankly, I’ve also tried to dig deeper by using prints all around and reached the same insights (But didn’t know how to solve it by using partial). Your skills are so valuable and impressive just so you know!

The clean solution that you offered was amazing in several ways:

It literally does what it’s required - enabling the user to pass in kwargs from the ImageBlock level
For me, it saved resources and computing time, because this way I didn’t need any longer to use Regex commands in order to open the labeled files
It helps users now to configure whatever parameters that they would want their RAW files to be post-processed now before the training part. This scalability is what I wanted to achieve and give back to this community.

You’re among the ones that very contributed much to this little project that will hopefully be added to fastai eventually this way or another. It’s like you added the last brick to the wall.

Danrohn · July 20, 2022, 1:03am

Not that my opinion is going to change the world, but while there are many conventions in practical coding, I personally like the WordWordWord version better rather than the word_word_word one for all uses, even if there’s such a convention where the first stands for methods/classes and the latter for functions/etc…