Ben I’ve had a look through your notebook (which is great by the way) and I have a weird idea.
remember those slides of the first few layers of resnet how they were just dots, tiny lines, gradients, etc… (I’m sorry I can’t lay my hands on the slide right now but you know the one i mean right?)
Looking at your galaxy images they are all exactly the same on a granular level, i.e. all stars. perhaps you could run them through some sort of filter and see what you can get. I’m thinking photoshops fast masks, unsharp masks, gradients, etc… I’m sorry I don’t know enough of the image modification libraries in Python to suggest one.