Deep Learning on X ray images


I am interested in interpreting X ray images using deep learning. Particularly Chest X ray images. I am using Montgomery TB dataset and Shenzhen TB dataset for this experiment. I am trying to predict whether an X ray image is normal or has TB (nodule) indicaitons.

I tried fine-tuning the ResNet models from lesson 1. So far the performance is not that great. ResNet on untreated images (converting gray to 3 channels) gives me accuracy in range of 68-70%. I tried histogram equalisation on X ray image and accuracy was in the range of 74-75%. With histogram equalisation as one of the channels of the image, the accuracy was in the range of 77-82%. Accuracies on validation set is comparable to training set accuracies.

  1. Are there any data preparation methods which are well suited for X ray images?
  2. Are there any different models which I should be trying?
  3. Are Imagenet models capable of extracting patterns from X ray images? If not, should I build my own model rather than using Imagenet models?
  4. The total set of images are 800 in number. Should I be doing data augmentation? What sort augmentation helps? (I can think of flipping the image on the horizontal axis and small random zooming in.)

Thanks in advance.

1 Like

The techniques for X ray and CT are quite similar, except X ray is a bit easier since only 2d. Therefore, I’d suggest checking out the kernels, solutions, etc to the 2017 Data Science Bowl, which used CT scans.


Thanks @jeremy

Check this out -


I wonder if pre-training on medical datasets like this would help for un-related problems. I just feel like ImageNet doesn’t perform super well for non-rgb images (understandably) and since this dataset doesn’t have any rgb images it may help if the classification problem is for non-rgb images even if its for classifying something like icebergs vs ships.

I am not sure if there is any learning that is transferable from xrays to some other non rgb data set such as icebergs, the high level visual features are very different.

did you get anything working on this? Looking at a few medical imaging datasets now.