Faimed3d - fastai extension for volumetric medical data

Hi all,

I work as a radiologist and used fastai for some research projects. Although fastai now supports medical images, I found that it is still very challenging to apply it to 3d data, such as CT or MRI datasets. So, over the last months I wrote faimed3d, an extension to fastai, which facilitates working with 3d data.
Currently, it is possible to train 3d ResNets and 3d U-Nets using the library. It supports a variety of medical formats including DICOM, DICOM series, NIfTI and more.
I already used the library in some of my research projects and aim to further improve the functionality.
You find the GitHub repository at https://github.com/kbressem/faimed3d and the docs at https://kbressem.github.io/faimed3d/ .
I would be happy about any feedback.

11 Likes

This is really cool, what would be great is a tutorial notebook (maybe something from your research projects if you can share) . I had also started experimenting on extensions to the fastai.medical module but not tailored to 3D here

1 Like

Thanks.
I provide two example on GitHub. They are on classification using the Stanford MR-NET dataset and on segmentation using the Coronacases dataset. Examples for 4D data will come soon.

I’ll also check out you extension, it look very nice.

1 Like

Thanks!, I see the examples now, cheers for sharing this.

I have updated faimed3d.

Pretrained 3D Models

faimed3d now provides some custom implementations of 3D ResNets and 3D EfficientNets, all pretrained on the UCF-101 dataset for transfer learning. Although the models do not surpass state-of-the-art, they come close with ResNet3D-101 scoring an equivalent of 17th place in the global leaderboard for models without additional training data.

DeepLab

I implemented a Dynamic-UNet-like version of DeepLab for 3D data, which supports all pretrained video models from torchvision and faimed3d as a backbone. DeepLab requires less GPU memory, which is crucial when processing large volumetric data and performs equally for segmentation (at least in my experiments).

DICOM Explorer

Viewing images is very important to understand the data, so faimed3d now provides a simple DICOM viewer implemented as an iPython widget. The viewer allows scrolling through a 3D volume similar to how a radiologist would do, supports windowing and overlay of segmentation masks.

It is now used as standard viewer for show_batch_3d

3 Likes