I am trying to do some segmentation task using fastai. I have several quite large images, like 6000x4000, which are very hard to be loaded to GPU. So I am thinking to split them into patches. A possible way is to use pytorch.unfold method, which is discussed here. My question is that is there any way to make a DataLoader-like interface for this operation? Transform seems to be a good fit for this task, but based on examples, transform seems to be one to one map (correct me if I am wrong). How to make a DataLoader that can split the large images on fly?
Hi longqi,
I think practically, preprocessing the images into small patches is usually good enough, as long as you don’t have a extreamly small dataset or the location of the patches matters. Based on your description, I assume you are dealing with a medical or remote sensing datasets? In this case, its commonly seen in papers that they cut patches with overlay.
I recently watched some fastai v2 walk-thru videos and I believe in fastai v2 (haven’t tried yet), splitting the images on the fly can be achieved by splitting the image into patches with a custom dataset class after loading it, and return the patches as tuples (like the siamese example), and finally batch them together in the after_batch callback. I remember these tuples are transformed independently so they are effectively
But one thing to note here is that if you cut it on the fly, the shuffling happens among whole images now instead of patches, so maybe you should experiment with your dataset to see if this (lack of) randomization affects training. Since your images are pretty big, I doubt you have enough GPUs to process an image in a single iteration, so you may need to throw some of the patches away to fit in GPU memory.
I’ll get into the new Pipeline API the next few days, if you are interested, I can give you more help after I watch more walk-thru videos 
In conclusion, I would recommend you just preprocess and save the patches.
Thanks a lot, @twofyw. I think you may be right, and I may need to the preprocessing in advance of training. After split to patches, they should be able to fit into fastai pipeline pretty smoothly. I am just curious whether there is a beautify and elegant way to do this kind of things inside fastai workflow.
A few people are working on the same thing here, and the posts include a bit of preliminary code. The hardest problem seems to be selecting random tiles + labels rather than whole images. Figuring out which label goes with which tile can also be a bit tricky; for example when bounding box crosses tile boundaries. My take on the current state of the art is that it’s easiest to stick with doing tiling (and re-assembly) outside of fastai…though it sure would be nice if someone figures out how to incorporate it.
In addition, this library should be handy for bounding box problems.
1 Like