Labels for image segmentation when we decreasing image size

In image segmentation when we are decreasing original image size for training, lets say by 2 for both dimension, what transformation we do for the ground truth labels for the pixels?
It seems that in this case each 4 pixels of the original image become a single pixel of the transformed image, what are we doing with labels of these pixels?

The targets is reduced in the same way, using nearest interpolation to determine the label of the pixel.

Thanks, but how can you do interpolation for the labels in this case?
A segmentation label should have a discrete value. And if you create 1 pixels from 4, you may have cases when all 4 of them had different segment labels or 2 has one and 2 other has second one.
How the algorithm will choose in this case which label to keep for a reduced image?

Any update on this @sgugger or @vladgets? It seems to limit the types of transforms that can be applied to ImageSegments for segmentation problems. Edited my reply because open_mask handles this correctly, image segment data should be stored internally as floats and the resampling methods can be found here

# image data (0-1 as float) can be transformed
img = Image(torch.randn(3, 224, 224))
# prints pixel data normally

# segmentation label data (0-n as long) cannot be transformed
label = ImageSegment(torch.randint(0, 10, (3, 224, 224)))
# RuntimeError: grid_sampler(): expected input and grid to have same dtype, but input has long long and grid has float

# segmentation label data should still be stored internally as float
# and then everything will work out nicely
label_proper = ImageSegment(torch.randint(0, 10, (3, 224, 224)).float())
# all good