hi,
i am wondering whether it is already possible to generate images in stable diffusion that have a higher dynamic range than 8 bit images? 32 bit would be ideal (but also with ranges that are not clamped to 0-1, but could go as high as 0-10 or 0-15? talking HDR imaging here.
i guess issues will be:
pretrained models only ‘know’ 8 bit images, so the model would have to be trained from scratch
PIL only allows 8 bits per channel
a wider range than 0-1 might cause issues?
i read that OpenCV allows wider bit depths, but i don’t see OpenCV mentioned much in this forum…
does anyone have some experience/insights to share?
thank you.
The models predictions are floating point numbers which are then translated into UInt8 range 0-256. There’s nothing stopping you from instead translating it to whatever range you want. ex: 0-1024 if you wanted 10 bit. You wouldn’t necessarily need (or want) to change the scale of the raw prediction range.
I believe SD was only trained with 8 bit images so I think it’s unlikely to produce results with the same range characteristics as if it were trained on higher precision input images, however I do think it would be an interesting experiment to convert raw outputs to a larger range and see if additional useful range could be extracted similar to a raw camera image. I actually do think there’s a decent chance there would be additional useful range, at least in some cases. I don’t think you’re going to get anywhere close to 32 bits of useful range due to how floating point numbers work and the fact that most likely you’re doing fp16 inference, but I think > 8bit seems plausible.
You wouldn’t need (or want) to retrain from scratch but instead fine-tune the model with higher dynamic range photos for the model to do a better job at better representing the larger ranges.
It does appear it supports larger ranges than PIL. No reason not to use it, it’s probably just going to be a bit less convenient than PIL.