Using 3D data

Part 1 (2019)
1

Hi,

I would like to create a 3D UNet for MRI segmentation (Multiple Sclerosis Lesions segmentation), but I can’t figure out how fastai will handle my 3D data.

Following this thread, it seems it is possible to use 3D data:

So I did the following:

from fastai import *
from fastai.vision import *
from fastai.callbacks.hooks import *
import nibabel as nib

def open_nifti(fn):
    image = nib.load(str(fn))
    return Image(torch.Tensor(image.get_fdata()))

class NiftiLabelList(SegmentationLabelList):
    "`ItemList` for segmentation nifti masks."
    def open(self, fn): return open_nifti(fn)
    
class NiftiItemList(SegmentationItemList):
    def open(self, fn): return open_nifti(fn)
    _label_cls = NiftiLabelList

bs = 8
src = (NiftiItemList.from_folder(path, extensions=('.nii'))
       .split_by_rand_pct()
       .label_from_func(imageToLabel, classes=codes))
data = (src.databunch(bs=bs))

def custom_acc(input, target):
    target = target.squeeze(1)
    return (input.argmax(dim=1)==target).float().mean()

metrics=custom_acc
wd=1e-2

learn = unet_learner(data, models.resnet34, metrics=metrics, wd=wd, bottle=True)

lr_find(learn, num_it=200)
learn.recorder.plot(suggestion=True)

but I got the following error:

...

~/fastai/lib/python3.7/site-packages/torch/nn/modules/conv.py in conv2d_forward(self, input, weight)
    340                             _pair(0), self.dilation, self.groups)
    341         return F.conv2d(input, weight, self.bias, self.stride,
--> 342                         self.padding, self.dilation, self.groups)
    343 
    344     def forward(self, input):

RuntimeError: Given groups=1, weight of size 64 3 7 7, expected input[8, 181, 217, 181] to have 3 channels, but got 181 channels instead

That does not surprise me since, from what I understood, the model models.resnet34 is designed for 2D data.

So why does it work in the above thread?

I would appreciate any help to work with 3D data in fastai :slight_smile:

2

DICOM is not necessarily 3D data.

Afaik, you should be able to set that up in fastai if you use a 3D model and 3D data loader as the rest should work similar.

3

Thx!
So I guess I need a 3D model; so my only option is to create one myself and train it from scratch, right?

4

You should find a pytorch model that is a suitable starting point. Try to have a look at the pytorch hub or at GitHub. :slight_smile:

5

All right, thanks you :slight_smile: