Hello,
I try to implement my custom PreProcessor by defining a Subclass inheriting from the BaseClass “PreProcessor” given in the data-block-API. In the creation of an ItemList, for testing purposes I tried to simply use the BaseClass “PreProcessor” as defined in the data-block-API, which should do noting but return the ItemList as given:
class PreProcessor():
"Basic class for a processor that will be applied to items at the end of the data block API."
def __init__(self, ds:Collection=None): self.ref_ds = ds
def process_one(self, item:Any): return item
def process(self, ds:Collection): ds.items = array([self.process_one(item) for item in ds.items])
In the creation of the ItemList (which works fine without the argument processor=PreProcessor) like
src=(ImageList.from_df(df=input_df, path='/data/',
cols='filename',processor=PreProcessor)
.split_by_rand_pct(0.2, seed=42)
.label_from_df(cols='class'))
when this PreProcessor is called after splitting and labelling, there is an error message about a missing positional argument in the processor-method. What is going wrong here? I have not done anything but use the BaseClass-PreProcessor, which I guess should work (even if it does nothing). If the BaseClass does not work, anything inheriting and not overwriting the process-method from it will also not work.
TypeError Traceback (most recent call last)
in
2 cols=‘filename’,processor=PreProcessor)
3 .split_by_rand_pct(0.2, seed=42)
----> 4 .label_from_df(cols=‘class’))
/usr/share/applications/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/data_block.py in _inner(*args, **kwargs)
478 self.valid = fv(*args, from_item_lists=True, **kwargs)
479 self.class = LabelLists
–> 480 self.process()
481 return self
482 return _inner
/usr/share/applications/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/data_block.py in process(self)
532 “Process the inner datasets.”
533 xp,yp = self.get_processors()
–> 534 for ds,n in zip(self.lists, [‘train’,‘valid’,‘test’]): ds.process(xp, yp, name=n)
535 #progress_bar clear the outputs so in some case warnings issued during processing disappear.
536 for ds in self.lists:
/usr/share/applications/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/data_block.py in process(self, xp, yp, name, max_warn_items)
712 p.warns = []
713 self.x,self.y = self.x[~filt],self.y[~filt]
–> 714 self.x.process(xp)
715 return self
716
/usr/share/applications/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/data_block.py in process(self, processor)
82 if processor is not None: self.processor = processor
83 self.processor = listify(self.processor)
—> 84 for p in self.processor: p.process(self)
85 return self
86
TypeError: process() missing 1 required positional argument: ‘ds’