NN tutorial for Pytorch


#1

Hi,
I’m trying to go over the great tutorial that the FastAI team just added to the Pytorch tutorial sitehttps://pytorch.org/tutorials/beginner/nn_tutorial.html.

I’m having hard time to understand the negative log-likeihood function, I understand from the text it is supposed to be standard Python broadcasting technique, but i’m not familiar how the range function behave inside a list comprehension.

def nll(input, target):
    return -input[range(target.shape[0]), target].mean()

Any help or a reference where to search for an explanation will be great.

Cheers


(youali) #2

Hi,

The loss function is L(y)=−log(y), which is summed for all the correct classes, and given that the model outputs are in the form of log probabilities, all we need to do is get the log probabilities for the correct classes, which is done by indexing the targets : input[range(target.shape[0]), target], So for each example in the batch (range(target.shape[0])) we get the outputs corresponding to the correct class, and we take the mean over the batch.

this is how I see it.


(MOHAMED AWNI HAMED) #3

could u please explain the line code with more details?