Loading RAW image files into ImageBlock

Hey guys,

Problem: How to load image files of *.RAW format into ImageBlock?

Suggested solution:

Looking at: (Data.py)

# Cell
def ImageBlock(cls=PILImage):
    "A `TransformBlock` for images of `cls`"
    return TransformBlock(type_tfms=cls.create, batch_tfms=IntToFloatTensor)

What if I can create another cls that is based on rawpy API?

So far it looks like that PIL API supports mostly *.JPG format.

This issue is followed by this issue:

Defining your own image class would be the right solution.
The following suggestion could work:


def load_raw_image(fn):
    "Open and load a raw image with rawpy"
    # your function to load a raw image from fn/Path here
    # return a numpy array `arr` 
    return arr


class RawpyImage(PILBase):
    @classmethod
    def create(cls, fn:(Path,str), **kwargs)->None:
        "Open a raw image from path `fn`"
        arr = load_raw_image(fn)
        return cls(Image.fromarray(arr))

Make sure the format of the np.array generated in load_raw_image is either uint8 (integer values between 0 and 255) or float32 (float values between 0 and 1).

1 Like

Thanks man! Sounds great.

Make sure the format of the np.array generated in load_raw_image is either uint8 (integer values between 0 and 255) or float32 (float values between 0 and 1).

JPG has about 8 bits for each pixel, so there are at most 256 different values. But RAW files have about 16 bits for each pixel, so there are at most 65536 different values. So would I need uint16, isn’t it?

In theory, yes, but IIRC, uint16 cannot be transformed to a PILImage instance. I had those problems when reading DICOM files, which are also 16 bit. Going with float values, rescaled to values between 0 and 1 might be easier.

So if I get it right, I should unpack the image file into a matrix, then divide every pixel by 65536, then convert it into np.array of the datatype of float32?

Yes. This should work.

1 Like

Hi !
I want to load 16 bits png images. Does fastai do this normalization for me?

Hi BresNet !
How does this fit in a simple image classification script? Where I call this class?

Maybe here:

ds = DataBlock(blocks = (ImageBlock(cls=RawpyImage),CategoryBlock),
                  get_items = get_image_files,
                  splitter = GrandparentSplitter(),
                  get_y = parent_label)

:thinking:

I’ve tried this:

def load_16bits_image(fn):
    img = Image.open(fn)
    arr = np.array(img) / 65365

    return arr


class Grayscale16bitsImage(PILBase):
    @classmethod
    def create(cls, fn : (Path, str), **kwargs)->None:
        "Open a raw image from path `fn`"
        arr = load_16bits_image(fn)
        return cls(Image.fromarray(arr))

path = Path('data')

item_tfms = [Resize(224)]

batch_tfms = aug_transforms(
    mult=2, do_flip=False, max_rotate=15
)

dblock = DataBlock(
    blocks=(ImageBlock(cls=Grayscale16bitsImage), CategoryBlock),
    get_items=get_image_files,
    splitter=GrandparentSplitter(train_name='train', valid_name='valid'),
    get_y=parent_label,
    item_tfms=item_tfms,
    batch_tfms=batch_tfms)

But I get the following error:

AttributeError: _tensor_cls
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-56-10885883025c> in <module>
     32 print(classes)
     33 
---> 34 dls_patches_one.show_batch()
     35 

~\Anaconda3\envs\dlenv\lib\site-packages\fastai\data\core.py in show_batch(self, b, max_n, ctxs, show, unique, **kwargs)
     98             old_get_idxs = self.get_idxs
     99             self.get_idxs = lambda: Inf.zeros
--> 100         if b is None: b = self.one_batch()
    101         if not show: return self._pre_show_batch(b, max_n=max_n)
    102         show_batch(*self._pre_show_batch(b, max_n=max_n), ctxs=ctxs, max_n=max_n, **kwargs)

~\Anaconda3\envs\dlenv\lib\site-packages\fastai\data\load.py in one_batch(self)
    146     def one_batch(self):
    147         if self.n is not None and len(self)==0: raise ValueError(f'This DataLoader does not contain any batches')
--> 148         with self.fake_l.no_multiproc(): res = first(self)
    149         if hasattr(self, 'it'): delattr(self, 'it')
    150         return res

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\basics.py in first(x, f, negate, **kwargs)
    545     x = iter(x)
    546     if f: x = filter_ex(x, f=f, negate=negate, gen=True, **kwargs)
--> 547     return next(x, None)
    548 
    549 # Cell

~\Anaconda3\envs\dlenv\lib\site-packages\fastai\data\load.py in __iter__(self)
    107         self.before_iter()
    108         self.__idxs=self.get_idxs() # called in context of main process (not workers/subprocesses)
--> 109         for b in _loaders[self.fake_l.num_workers==0](self.fake_l):
    110             if self.device is not None: b = to_device(b, self.device)
    111             yield self.after_batch(b)

~\Anaconda3\envs\dlenv\lib\site-packages\torch\utils\data\dataloader.py in __next__(self)
    519             if self._sampler_iter is None:
    520                 self._reset()
--> 521             data = self._next_data()
    522             self._num_yielded += 1
    523             if self._dataset_kind == _DatasetKind.Iterable and \

~\Anaconda3\envs\dlenv\lib\site-packages\torch\utils\data\dataloader.py in _next_data(self)
    559     def _next_data(self):
    560         index = self._next_index()  # may raise StopIteration
--> 561         data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
    562         if self._pin_memory:
    563             data = _utils.pin_memory.pin_memory(data)

~\Anaconda3\envs\dlenv\lib\site-packages\torch\utils\data\_utils\fetch.py in fetch(self, possibly_batched_index)
     32                 raise StopIteration
     33         else:
---> 34             data = next(self.dataset_iter)
     35         return self.collate_fn(data)
     36 

~\Anaconda3\envs\dlenv\lib\site-packages\fastai\data\load.py in create_batches(self, samps)
    116         if self.dataset is not None: self.it = iter(self.dataset)
    117         res = filter(lambda o:o is not None, map(self.do_item, samps))
--> 118         yield from map(self.do_batch, self.chunkify(res))
    119 
    120     def new(self, dataset=None, cls=None, **kwargs):

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\basics.py in chunked(it, chunk_sz, drop_last, n_chunks)
    214     if not isinstance(it, Iterator): it = iter(it)
    215     while True:
--> 216         res = list(itertools.islice(it, chunk_sz))
    217         if res and (len(res)==chunk_sz or not drop_last): yield res
    218         if len(res)<chunk_sz: return

~\Anaconda3\envs\dlenv\lib\site-packages\fastai\data\load.py in do_item(self, s)
    131     def prebatched(self): return self.bs is None
    132     def do_item(self, s):
--> 133         try: return self.after_item(self.create_item(s))
    134         except SkipItemException: return None
    135     def chunkify(self, b): return b if self.prebatched else chunked(b, self.bs, self.drop_last)

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\transform.py in __call__(self, o)
    198         self.fs = self.fs.sorted(key='order')
    199 
--> 200     def __call__(self, o): return compose_tfms(o, tfms=self.fs, split_idx=self.split_idx)
    201     def __repr__(self): return f"Pipeline: {' -> '.join([f.name for f in self.fs if f.name != 'noop'])}"
    202     def __getitem__(self,i): return self.fs[i]

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\transform.py in compose_tfms(x, tfms, is_enc, reverse, **kwargs)
    148     for f in tfms:
    149         if not is_enc: f = f.decode
--> 150         x = f(x, **kwargs)
    151     return x
    152 

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\transform.py in __call__(self, x, **kwargs)
     71     @property
     72     def name(self): return getattr(self, '_name', _get_name(self))
---> 73     def __call__(self, x, **kwargs): return self._call('encodes', x, **kwargs)
     74     def decode  (self, x, **kwargs): return self._call('decodes', x, **kwargs)
     75     def __repr__(self): return f'{self.name}:\nencodes: {self.encodes}decodes: {self.decodes}'

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\transform.py in _call(self, fn, x, split_idx, **kwargs)
     81     def _call(self, fn, x, split_idx=None, **kwargs):
     82         if split_idx!=self.split_idx and self.split_idx is not None: return x
---> 83         return self._do_call(getattr(self, fn), x, **kwargs)
     84 
     85     def _do_call(self, f, x, **kwargs):

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\transform.py in _do_call(self, f, x, **kwargs)
     88             ret = f.returns(x) if hasattr(f,'returns') else None
     89             return retain_type(f(x, **kwargs), x, ret)
---> 90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)
     92 

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\transform.py in <genexpr>(.0)
     88             ret = f.returns(x) if hasattr(f,'returns') else None
     89             return retain_type(f(x, **kwargs), x, ret)
---> 90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)
     92 

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\transform.py in _do_call(self, f, x, **kwargs)
     87             if f is None: return x
     88             ret = f.returns(x) if hasattr(f,'returns') else None
---> 89             return retain_type(f(x, **kwargs), x, ret)
     90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)

~\Anaconda3\envs\dlenv\lib\site-packages\fastcore\dispatch.py in __call__(self, *args, **kwargs)
    116         elif self.inst is not None: f = MethodType(f, self.inst)
    117         elif self.owner is not None: f = MethodType(f, self.owner)
--> 118         return f(*args, **kwargs)
    119 
    120     def __get__(self, inst, owner):

~\Anaconda3\envs\dlenv\lib\site-packages\fastai\vision\core.py in encodes(self, o)
    221 # Cell
    222 @ToTensor
--> 223 def encodes(self, o:PILBase): return o._tensor_cls(image2tensor(o))
    224 @ToTensor
    225 def encodes(self, o:PILMask): return o._tensor_cls(image2tensor(o)[0])

~\Anaconda3\envs\dlenv\lib\site-packages\PIL\Image.py in __getattr__(self, name)
    544             )
    545             return self._category
--> 546         raise AttributeError(name)
    547 
    548     @property

AttributeError: _tensor_cls

Maybe load_16bits_image(fn) isn’t loading correctly the images here img = Image.open(fn). How can I see the values of fn?

1 Like

My guess would be, too, that the error is in load_16bits_image_().
Try loading a file with the function only and check if you get a valid numpy array.

arr = load_16bits_image_(`path/to/file`)
arr.shape, arr.max(), arr.min()

If that works, maybe the error is in your getters. Does get_image_files return a valid file name? you could use print debugging in load_16bits_image_() function. This way, you’ll know what the values of fn are. It should be a valid FileName.

I tried to implement it now. I’ve got this far:

def loadRawImage(fn):
    fn=str(fn)
    raw_file=rawpy.imread(fn)
    raw_converted_file=raw_file.raw_image_visible.astype(np.float32)
    ndarr=np.asarray(raw_converted_file)/(65536.0)
    return ndarr

class RawpyImage(PILBase):
    @classmethod
    def create(cls, fn:(Path, str), **kwargs)->None:
        "Open a raw image from path `fn`"
        arr = loadRawImage(fn)
        return cls(Image.fromarray(arr, mode="RGB"))

    def __repr__(self): return f'{self.__class__.__name__} mode={self.mode} size={"x".join([str(d) for d in self.size])}'

But then I got this message, which I can’t tell where it comes from: (I tried to find this error type in Rawpy package, but there isn’t that I could find)

LibRawIOError                             Traceback (most recent call last)
<ipython-input-54-4ad80883aa74> in <module>()
     ...
      8                    )
----> 9 dls = dblock.dataloaders(source=path,bs=btchSize,path=path)
     ...

/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in dataloaders(self, source, path, verbose, **kwargs)
    111 
    112     def dataloaders(self, source, path='.', verbose=False, **kwargs):
--> 113         dsets = self.datasets(source, verbose=verbose)
    114         kwargs = {**self.dls_kwargs, **kwargs, 'verbose': verbose}
    115         return dsets.dataloaders(path=path, after_item=self.item_tfms, after_batch=self.batch_tfms, **kwargs)

/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in datasets(self, source, verbose)
    108         splits = (self.splitter or RandomSplitter())(items)
    109         pv(f"{len(splits)} datasets of sizes {','.join([str(len(s)) for s in splits])}", verbose)
--> 110         return Datasets(items, tfms=self._combine_type_tfms(), splits=splits, dl_type=self.dl_type, n_inp=self.n_inp, verbose=verbose)
    111 
    112     def dataloaders(self, source, path='.', verbose=False, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in __init__(self, items, tfms, tls, n_inp, dl_type, **kwargs)
    326     def __init__(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
    327         super().__init__(dl_type=dl_type)
--> 328         self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
    329         self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
    330 

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in <listcomp>(.0)
    326     def __init__(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
    327         super().__init__(dl_type=dl_type)
--> 328         self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
    329         self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
    330 

/usr/local/lib/python3.7/dist-packages/fastcore/foundation.py in __call__(cls, x, *args, **kwargs)
     95     def __call__(cls, x=None, *args, **kwargs):
     96         if not args and not kwargs and x is not None and isinstance(x,cls): return x
---> 97         return super().__call__(x, *args, **kwargs)
     98 
     99 # Cell

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in __init__(self, items, tfms, use_list, do_setup, split_idx, train_setup, splits, types, verbose, dl_type)
    252         if do_setup:
    253             pv(f"Setting up {self.tfms}", verbose)
--> 254             self.setup(train_setup=train_setup)
    255 
    256     def _new(self, items, split_idx=None, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in setup(self, train_setup)
    274             for f in self.tfms.fs:
    275                 self.types.append(getattr(f, 'input_types', type(x)))
--> 276                 x = f(x)
    277             self.types.append(type(x))
    278         types = L(t if is_listy(t) else [t] for t in self.types).concat().unique()

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in __call__(self, x, **kwargs)
     71     @property
     72     def name(self): return getattr(self, '_name', _get_name(self))
---> 73     def __call__(self, x, **kwargs): return self._call('encodes', x, **kwargs)
     74     def decode  (self, x, **kwargs): return self._call('decodes', x, **kwargs)
     75     def __repr__(self): return f'{self.name}:\nencodes: {self.encodes}decodes: {self.decodes}'

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _call(self, fn, x, split_idx, **kwargs)
     81     def _call(self, fn, x, split_idx=None, **kwargs):
     82         if split_idx!=self.split_idx and self.split_idx is not None: return x
---> 83         return self._do_call(getattr(self, fn), x, **kwargs)
     84 
     85     def _do_call(self, f, x, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _do_call(self, f, x, **kwargs)
     87             if f is None: return x
     88             ret = f.returns(x) if hasattr(f,'returns') else None
---> 89             return retain_type(f(x, **kwargs), x, ret)
     90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)

/usr/local/lib/python3.7/dist-packages/fastcore/dispatch.py in __call__(self, *args, **kwargs)
    116         elif self.inst is not None: f = MethodType(f, self.inst)
    117         elif self.owner is not None: f = MethodType(f, self.owner)
--> 118         return f(*args, **kwargs)
    119 
    120     def __get__(self, inst, owner):

<ipython-input-43-c9b93e6bca6e> in create(cls, fn, **kwargs)
     15     def create(cls, fn:(Path, str), **kwargs)->None:
     16         "Open a raw image from path `fn`"
---> 17         arr = loadRawImage(fn)
     18         return cls(Image.fromarray(arr, mode="RGB"))
     19 

<ipython-input-43-c9b93e6bca6e> in loadRawImage(fn)
      6     #print(fn.encode())
      7     #print(str(fn))
----> 8     raw_file=rawpy.imread(fn)
      9     raw_converted_file=raw_file.raw_image_visible.astype(np.float32)
     10     ndarr=np.asarray(raw_converted_file)/(65536.0)

/usr/local/lib/python3.7/dist-packages/rawpy/__init__.py in imread(pathOrFile)
     18         d.open_buffer(pathOrFile)
     19     else:
---> 20         d.open_file(pathOrFile)
     21     return d

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.open_file()

rawpy/_rawpy.pyx in rawpy._rawpy.RawPy.handle_error()

LibRawIOError: b'Input/output error'

So hell what, I don’t get it where or what does it mean:

LibRawIOError: b'Input/output error'

Any ideas?

Maybe your file paths are not correct? Or the file format is not compatible with rawpy?

I tried that function on some files somewhere else, and the function did work.
Apparently, it doesn’t work only when the DataBlock starts running it, but I can’t figure out how to debug this correctly… Because I didn’t find where it prints “b’Input/output error’”

OK, I found the solution for this error.

So what happened basically was: this error

LibRawIOError: b'Input/output error'

Told me that this command (see below) couldn’t work:

d.open_file(pathOrFile)

Which means that it couldn’t open the file. This could be because the file wasn’t in the folder, or because the path was wrong, or perhaps some other option.

I just kept adding print() at every possible step to see what happens, until I finally found out that at some point, that command was fed by a wrong auto-generated path (of the target files). I just had to change that auto-generating function, so it creates the path with the correct new format (of RAW files rather than JPG files).

So it worked.
But now I’m facing other problems, Yay! Oh wait, you guys are gonna love them. So fun them the new ones.

Great job man!

Ok, funny fact is that I encountered the same error as yours!

Here I can also track the log to see what went wrong.

Basically, the error claims that the new class that we created doesn’t include a field named “_tensor_cls”.

But checking this out, even PILBase doesn’t have that field. So I’m not even sure where I should add it myself…

Setting-up type transforms pipelines
Collecting items from /content/drive/MyDrive/Dataset12/train/Short
Found 1 items
2 datasets of sizes 1,0
Setting up Pipeline: RawpyImage.create
/content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
Setting up Pipeline: my_get_y -> RawpyImage.create
/content/drive/MyDrive/Dataset12/train/Long/P1200131.ORF

Building one sample
  Pipeline: RawpyImage.create
    starting from
      /content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
/content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
    applying RawpyImage.create gives
      RawpyImage mode=RGB size=5240x3912
  Pipeline: my_get_y -> RawpyImage.create
    starting from
      /content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
    applying my_get_y gives
      /content/drive/MyDrive/Dataset12/train/Long/P1200131.ORF
/content/drive/MyDrive/Dataset12/train/Long/P1200131.ORF
    applying RawpyImage.create gives
      RawpyImage mode=RGB size=5240x3912
/content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
/content/drive/MyDrive/Dataset12/train/Long/P1200131.ORF

Final sample: (RawpyImage mode=RGB size=5240x3912, RawpyImage mode=RGB size=5240x3912)


Collecting items from /content/drive/MyDrive/Dataset12/train/Short
Found 1 items
2 datasets of sizes 1,0
Setting up Pipeline: RawpyImage.create
/content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
Setting up Pipeline: my_get_y -> RawpyImage.create
/content/drive/MyDrive/Dataset12/train/Long/P1200131.ORF
Setting up after_item: Pipeline: ToTensor
Setting up before_batch: Pipeline: 
Setting up after_batch: Pipeline: IntToFloatTensor -- {'div': 255.0, 'div_mask': 1}
/content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
/content/drive/MyDrive/Dataset12/train/Long/P1200131.ORF
Could not do one pass in your dataloader, there is something wrong in it

Building one batch
Applying item_tfms to the first sample:
/content/drive/MyDrive/Dataset12/train/Short/P1200133.ORF
/content/drive/MyDrive/Dataset12/train/Long/P1200131.ORF
  Pipeline: ToTensor
    starting from
      (RawpyImage mode=RGB size=5240x3912, RawpyImage mode=RGB size=5240x3912)
    applying ToTensor failed.
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-34-585fa9d83a7f> in <module>()
----> 1 dblock.sum()

9 frames
/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in summary(self, source, bs, show_batch, **kwargs)
    168     if len([f for f in dls.train.after_item.fs if f.name != 'noop'])!=0:
    169         print("Applying item_tfms to the first sample:")
--> 170         s = [_apply_pipeline(dls.train.after_item, dsets.train[0])]
    171         print(f"\nAdding the next {bs-1} samples")
    172         s += [dls.train.after_item(dsets.train[i]) for i in range(1, bs)]

/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in _apply_pipeline(p, x)
    137         except Exception as e:
    138             print(f"    applying {name} failed.")
--> 139             raise e
    140     return x
    141 

/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in _apply_pipeline(p, x)
    133         name = f.name
    134         try:
--> 135             x = f(x)
    136             if name != "noop": print(f"    applying {name} gives\n      {_short_repr(x)}")
    137         except Exception as e:

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in __call__(self, x, **kwargs)
     71     @property
     72     def name(self): return getattr(self, '_name', _get_name(self))
---> 73     def __call__(self, x, **kwargs): return self._call('encodes', x, **kwargs)
     74     def decode  (self, x, **kwargs): return self._call('decodes', x, **kwargs)
     75     def __repr__(self): return f'{self.name}:\nencodes: {self.encodes}decodes: {self.decodes}'

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _call(self, fn, x, split_idx, **kwargs)
     81     def _call(self, fn, x, split_idx=None, **kwargs):
     82         if split_idx!=self.split_idx and self.split_idx is not None: return x
---> 83         return self._do_call(getattr(self, fn), x, **kwargs)
     84 
     85     def _do_call(self, f, x, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _do_call(self, f, x, **kwargs)
     88             ret = f.returns(x) if hasattr(f,'returns') else None
     89             return retain_type(f(x, **kwargs), x, ret)
---> 90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)
     92 

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in <genexpr>(.0)
     88             ret = f.returns(x) if hasattr(f,'returns') else None
     89             return retain_type(f(x, **kwargs), x, ret)
---> 90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)
     92 

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _do_call(self, f, x, **kwargs)
     87             if f is None: return x
     88             ret = f.returns(x) if hasattr(f,'returns') else None
---> 89             return retain_type(f(x, **kwargs), x, ret)
     90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)

/usr/local/lib/python3.7/dist-packages/fastcore/dispatch.py in __call__(self, *args, **kwargs)
    116         elif self.inst is not None: f = MethodType(f, self.inst)
    117         elif self.owner is not None: f = MethodType(f, self.owner)
--> 118         return f(*args, **kwargs)
    119 
    120     def __get__(self, inst, owner):

/usr/local/lib/python3.7/dist-packages/fastai/vision/core.py in encodes(self, o)
    221 # Cell
    222 @ToTensor
--> 223 def encodes(self, o:PILBase): return o._tensor_cls(image2tensor(o))
    224 @ToTensor
    225 def encodes(self, o:PILMask): return o._tensor_cls(image2tensor(o)[0])

AttributeError: 'RawpyImage' object has no attribute '_tensor_cls'

Alright guys, I managed to solve this issue by adding _tensor_cls to RawpyImage. It took me some nerve to dive into the source and search for those attributes and other issues.

But then I realized something: The whole fastai library is based on the PIL library for images files, which itself does not support any RAW files. So adding this RawpyImage class isn’t enough, as PILBase inherits from Image.Image (PIL).

Meaning, adding the RawpyImage class, that inherits from PILBase, that in turn inherits from PIL, makes just no sense - because PIL does not support RAW Files. All of the rest of Fastai classes and methods, such as “show_image” or “RandomResizedCrop”, are all based on getting PIL objects… Not Rawpy files…

I’ve come up to the thought that an approach change is needed here.

@jeremy

I’d love to hear your opinion about this, and maybe a little guidance for me, as maybe it’s quite easy to solve and contribute to the project.

My whole need is to have the framework of fastai support Rawpy objects. So while PIL objects are being converted into Tensor objects (of Pytorch), maybe it would be even still easy to add another layer of support, by making fastai convert Rawpy objects into Tesnor objects, so the bones of the fasiai framework still apply to those tensors for training it.

Hey folks,

I’m trying to rebuild the PILBase as RAWBase, so it will support RAW files.

What do you think could be the problem here?

My code:

class RawBase(Image.Image, metaclass=BypassNewMeta): #, metaclass=BypassNewMeta
    "This is to copy class PILBase"
    _bypass_type=Image.Image
    _show_args = {'cmap':'viridis'}
    _open_args = {'mode': 'RGB'}
    @classmethod 
    #staticmethod
    def create(cls,fn:(Path,str), *args, **kwargs)->None:
        return cls(load_raw_image(fn, *args, **kwargs))

    def __repr__(self): return f'{self.__class__.__name__} mode={self.mode} size={"x".join([str(d) for d in self.size])}'

Of course, RAWBase won’t inherit Image.Image eventually, because that is also part of PIL library. But there’s an error that I can’t tell where it comes from or why it does.

/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in dataloaders(self, source, path, verbose, **kwargs)
    111 
    112     def dataloaders(self, source, path='.', verbose=False, **kwargs):
--> 113         dsets = self.datasets(source, verbose=verbose)
    114         kwargs = {**self.dls_kwargs, **kwargs, 'verbose': verbose}
    115         return dsets.dataloaders(path=path, after_item=self.item_tfms, after_batch=self.batch_tfms, **kwargs)

/usr/local/lib/python3.7/dist-packages/fastai/data/block.py in datasets(self, source, verbose)
    108         splits = (self.splitter or RandomSplitter())(items)
    109         pv(f"{len(splits)} datasets of sizes {','.join([str(len(s)) for s in splits])}", verbose)
--> 110         return Datasets(items, tfms=self._combine_type_tfms(), splits=splits, dl_type=self.dl_type, n_inp=self.n_inp, verbose=verbose)
    111 
    112     def dataloaders(self, source, path='.', verbose=False, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in __init__(self, items, tfms, tls, n_inp, dl_type, **kwargs)
    327     def __init__(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
    328         super().__init__(dl_type=dl_type)
--> 329         self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
    330         self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
    331 

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in <listcomp>(.0)
    327     def __init__(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
    328         super().__init__(dl_type=dl_type)
--> 329         self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
    330         self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
    331 

/usr/local/lib/python3.7/dist-packages/fastcore/foundation.py in __call__(cls, x, *args, **kwargs)
     95     def __call__(cls, x=None, *args, **kwargs):
     96         if not args and not kwargs and x is not None and isinstance(x,cls): return x
---> 97         return super().__call__(x, *args, **kwargs)
     98 
     99 # Cell

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in __init__(self, items, tfms, use_list, do_setup, split_idx, train_setup, splits, types, verbose, dl_type)
    253         if do_setup:
    254             pv(f"Setting up {self.tfms}", verbose)
--> 255             self.setup(train_setup=train_setup)
    256 
    257     def _new(self, items, split_idx=None, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastai/data/core.py in setup(self, train_setup)
    275             for f in self.tfms.fs:
    276                 self.types.append(getattr(f, 'input_types', type(x)))
--> 277                 x = f(x)
    278             self.types.append(type(x))
    279         types = L(t if is_listy(t) else [t] for t in self.types).concat().unique()

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in __call__(self, x, **kwargs)
     71     @property
     72     def name(self): return getattr(self, '_name', _get_name(self))
---> 73     def __call__(self, x, **kwargs): return self._call('encodes', x, **kwargs)
     74     def decode  (self, x, **kwargs): return self._call('decodes', x, **kwargs)
     75     def __repr__(self): return f'{self.name}:\nencodes: {self.encodes}decodes: {self.decodes}'

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _call(self, fn, x, split_idx, **kwargs)
     81     def _call(self, fn, x, split_idx=None, **kwargs):
     82         if split_idx!=self.split_idx and self.split_idx is not None: return x
---> 83         return self._do_call(getattr(self, fn), x, **kwargs)
     84 
     85     def _do_call(self, f, x, **kwargs):

/usr/local/lib/python3.7/dist-packages/fastcore/transform.py in _do_call(self, f, x, **kwargs)
     87             if f is None: return x
     88             ret = f.returns(x) if hasattr(f,'returns') else None
---> 89             return retain_type(f(x, **kwargs), x, ret)
     90         res = tuple(self._do_call(f, x_, **kwargs) for x_ in x)
     91         return retain_type(res, x)

/usr/local/lib/python3.7/dist-packages/fastcore/dispatch.py in __call__(self, *args, **kwargs)
    121         elif self.inst is not None: f = MethodType(f, self.inst)
    122         elif self.owner is not None: f = MethodType(f, self.owner)
--> 123         return f(*args, **kwargs)
    124 
    125     def __get__(self, inst, owner):

<ipython-input-37-5c1b39903303> in create(cls, fn, *args, **kwargs)
     36     #staticmethod
     37     def create(cls,fn:(Path,str), *args, **kwargs)->None:
---> 38         return cls(load_raw_image(fn, *args, **kwargs))
     39 
     40     def show(self):

/usr/local/lib/python3.7/dist-packages/fastcore/meta.py in __call__(cls, x, *args, **kwargs)
     60         if hasattr(cls, '_new_meta'): x = cls._new_meta(x, *args, **kwargs)
     61         elif not isinstance(x,getattr(cls,'_bypass_type',object)) or len(args) or len(kwargs):
---> 62             x = super().__call__(*((x,)+args), **kwargs)
     63         if cls!=x.__class__: x.__class__ = cls
     64         return x

TypeError: __init__() takes 1 positional argument but 2 were given

Can you please show us how come float32 did work for you?
I tried your way once again, and it seems that Image.fromarray() will turn the float32 ndarray into uint8 type of image object whatsoever.

It’s been a while, since I’ve done this, so I don’t remember exactly how I did it. You can try to change the mode of the image, fromarray has an argument for this.
Is there a specific reason, you need the array to be a PIL.Image? I believe all augmentations also work with numpy arrays or tensors?

Hey man! Oh.
I’ve tried to adjust fastai to support RAW files, but it seems that a lot of the other vision methods (such as “show” and augmentations) are based on Image.Image objects. I may be wrong (hopefully I am), and that’s why I was eager to find a way to only adjust the PILBase class to support RAW.

I found out that PIL supports TIFF and BMP formats, which also share 32bits and 16bits,24bits accordingly, so I was thinking about maybe I’ll just load the RAW files as converted to BMP files.

But it turns out that PIL does support BMP, meaning that it reads it and gets its attributes, but it stores the values of the pixels in uint8 type (after scaling). That’s just a huge bummer.

So yes, I will have to rewrite a whole class just for RAW files. But I can’t tell whether the rest of fastai will support the class augumentations methods.