Trying to understand loss function and filters in NN

Part 1 (2018)
1

Hey community, hope you can help me with some of the fundamental concepts in NN

My understanding of NN is,

  1. NN is one giant multi-dimensional loss function with the loss on one axis and the weights on the other axes
  2. Initially, the weights are assigned random values
  3. When you train a model what is essentially happening is each input (eg. image), from the training set, is processed through a funnel of different arrays of weights from the loss function (I am guessing these arrays are filters), to get to a prediction.
  4. Layers in the NN are a way to group filters (array of weights)
  5. The prediction on an input is checked against the true value of that input to calculate the loss for a given set of weights
  6. SGD technique is being used to navigate the loss function to arrive at a loss value that is lower than the previous loss value for a given set of weights, this is how weights are tuned.

Hope I am on the right track here :slight_smile:

Based on my understanding, I am trying to figure out the following,

  1. What is this giant multi-dimensional loss function?
  2. What are the filters? Are they indeed just arrays of weights defined in the loss function? How are these arrays created?
1 Like
2

Yes that’s all quite correct! The loss function is equal to the formulae shown in the XL convolution spreadsheet (which also shows examples of filters - although they’re randomly generated initially in practice), with a softmax function applied to the final set of activations (which is shown in a separate XL spreadsheet), and which is finally compared to the labels using negative log likelihood (which we haven’t covered in the DL course yet, but we did cover in the most recent less of the ML course).

3

Thanks @jeremy!