FocalLossFlat(): reasonable minimum for gamma?

Unet / ResNet for image segmentation.

I’m doing hyperparameter optimization with Optuna. One of the hyperparameters is the gamma factor for FocalLossFlat. I’ve set a very low minimum value for it: 1e-10. Based on a few trials so far, when gamma is very low, I get NaN in the validation loss.

What could be considered a “safe” minimum value for gamma, that would not waste time generating NaN in validation?

Ideally I would like to explore low values for this parameter, since gamma=0 is equivalent to CrossEntropyLoss. But at the same time I need to make sure I’m not wasting time.

So, any suggestions for the lowest safe value of gamma?