these are my functions…i have used the same code as in the original notebook but somehow my
nll is coming inf… I have no idea why this is happening.
these are my functions…i have used the same code as in the original notebook but somehow my
Hello,
If one of the probabilities that are computed via the softmax function is 0, because the logarithm of 0 is negative infinity, your loss would diverge to infinity. Could you please check what the minimum value in sm_pred is (i.e., sm_pred.min())?
Please note that in the nll function, there is negation happening, and therefore negative infinity would turn into infinity.
Thank you.
yeah, the min value of sm_pred is -inf
Hello,
Since the minimum value of sm_pred is negative infinity, that would indicate the magnitudes of the values in your logits (that is, pred) are excessively large. Because I am not aware of your full training pipeline, I can’t precisely pin down the source of this problem, however, based on personal experience, a common culprit for such an issue is too high a learning rate.
If you’d like, you could share more information regarding your code, and I’d be happy to help where I can.
2 Likes
Hello,
Are you receiving negative infinity immediately after initializing your network or after training it? Also, please use nn.ModuleList rather than Python’s standard list inside the Model class, and replace __call__ with forward.
1 Like
Notebook
This is the notebook from the course that i am reproducing… this is the first instance of the Model class… nn.ModuleList is used later in the notebook, that solves this problem. But i want to know why is this happening when i am using the exact code.
Hello,
That’s strange; you are running the notebook line by line and getting this issue? I executed it myself and encountered no such problem. Could you restart the notebook and run everything in order again?
Thank you.
The only difference between the original notebook and my notebook is that i am using the dataset from the kaggle competition digit-recognizer. I am hoping the problem is not because of the dataset?
Hello,
Apologies for the late reply. The dataset itself is not to blame per se, however, an issue might be the scale of the data. Could you please print out a random sample from the dataset, as well as its range (e.g., print(data.min(), data.max()))? I ask because it’s possible that the data falls in the range [0, 255], whereas PyTorch expects it to be normalized by a factor of 255.
Thank you.
2 Likes
