I have been training on my custom dataset for months and I just found out that I didn’t do any sorts of input normalization (in PyTorch, I did no normalization after ToTensor()). Could it be the cause that my network sometimes does not converge?
The problem is that I hope my network could work for different datasets (coming from different distribution), so I don’t know how to set a proper mean and std. What should be the best practice here?
Also my network is Resnet which supports Batch Norm. Is input normalization still important even if there is Batch Norm after every convolutional layer?
This is my first post in this forum and hopefully someone could provide me some generous hints Thanks in advance!