I’d like to train a neural network using MSE as loss function for a regression problem. However, my labels are log ratios of some gene change which are range [-5, 1]. To achieve a better accuracy, should I normalize labels to a balanced range like [-1,1]? Or it is totally fine to train a model from that range?
Hi. Typically you would leave the labels as is, and tack on a k1+sigmoid()*k2 layer at the end of the model that can only generate [-5,1] regardless of the input. I don’t think there is any well-defined theory behind this practice. “It makes it easier for the model,” in the sense that the original model is allowed to produce any value and it will still map to the label space. In practice, the underlying function may be learned faster and with more accuracy.
IDK how the log ratio would matter. If I were doing it, I’d experiment with and without sigmoid, and predicting the log ratio vs, the actual ratio.
Just some ideas and HTH,
1 Like
As @Pomo said, I regularly use the fastai layer called SigmoidRange([-5,1]) at the end of the network.
2 Likes