So if we view these decoded samples, then we can see exactly the images (with the valid transforms) that the predictions were run against instead of the original (from disk images) correct?
*I’m always a bit paranoid about how resizing has affected the images for predictions vs the original image as in my case with medical stuff, key data could have been clipped/lost during transformation and thus I want to view the ‘as the cnn sees it’ images.
Yup, only stuff that is undone is when your Transform has a decodes (like Normalize), but you will see crops/rotates/zooms… done to your image.
I’m not able to get into the images via dl.decode_batch() as I had hoped - would anyone have code showing this?
(I’m doing image segmentation, and want to see the results of the predicted mask on the original image as the CNN saw it…
Since the new images are random in size, but resized with padding, then when I overlay the mask back out to the original … if I can’t see the transformed image the random padding effect will end up being way off without doing a lot of math to figure out where the padding was used per image - i.e.
You mistake decode for something it does not: it undoes transforms that are critical to look at the data, like Categorize. It does not undo data augmentation transforms.
@sgugger - not sure why you assume that…I want to see the images with the augmentation, so my understanding is correct. You already clarified this point earlier (and thanks for that):
“Yup, only stuff that is undone is when your Transform has a decodes (like Normalize), but you will see crops/rotates/zooms… done to your image.”
My question is for a code example to show how to access the images directly, as the CNN would see them, so I can overlay the returned predicted mask properly and see how it’s performing 
Right now I get a mask from the predictions… but since the incoming images are from users, they were padded and resized to a fixed size for inference…thus, it’s hard to compare the predicted mask to the original image…much faster if we can overlay predicted mask with transformed image the prediction was made on to see results.
I am not sure I understand your problem. The inputs fed to the model are accessible with either the dataloader you use or by passing with_input=True to get_preds.
Problem (or use case really) is this:
a - user submits image for segmentation. The size of the image could be anything - 3048x 2046, 2040x1800, etc.
b - to do the prediction, we thus resize it to 1024x1024, with padding.
c - get predicted mask
d - we know have a mask for 1024x1024, but that only fits to the transformed/resized image, not the original. Thus to show results, want to access the transformed image and apply the mask for display.
Learn.show_results doesn’t show the results for test batch (I’m using test batch to do inference), it shows the latest training result.
So my question is how do we quickly access the predicted image with transforms to display a result of the inference?
If we take the predicted mask and apply it to the original image, even with interpolating it back out, it’s off b/c of the unknown padding…longer term we’ll have to do the math to fix this but for now just want to see quick results during training against hold out images for inference.
Thanks!
Again, the input of you dataloader is what you are looking for. Could you share some code as I really don’t understand your problem.
thanks - let me see if I can get the images via dataloader as you are mentioning. I thought the transforms wouldn’t fire until loaded for actual prediction (and thus the whole .decode() discussion above).
Otherwise yes I’ll share out a notebook.
Appreciate your help!
How do we pass from:
TensorMask -> PILMask
I want to be able to save on file the predicted masks from my segmentation model.
I think PILMask.create accepts a tensor.
I get his:
TypeError: Cannot handle this data type: (1, 1), <i8
I will propose a fix,.
I tried a day and it says that you need to pass into cpu, detactch it and call numpy
I have a really basic question on this topic. Once I’ve used get_preds to get the indices of the predictions, how do I find which of my labels the indices corresponded to? In fastai1 I think this was in learn.data.classes but this no longer seems to exist.
Yessssss thank you!