Hello
I’m trying to create a LSTM NN to predict weather variables (just a toy project). I implemented a custom loss function that penalizes with increased weight loss in recent dataset values: something like: mse(x) * index_of_x
the question is: should I use mse(x) * index_of_x in the training phase and mse(x) in the validation? my intuition says yes
Thank you
I’m just curious to know why you use index_of_x in the loss. How do you expect this to affect the training?
cause the index with higher index has more recent data. the index is a date
Interesting Idea.
You’re trying to predict the weather on a given day using the weather of a few earlier days. Wouldn’t using the date or number of days since the first recorded weather in the loss make the model dependent on the date?
Anyways, since you are using mse*index in the training phase, I feel it will be better to use the same for validation since mse*index is your loss metric.
I’m really interested to know how this works. When you’re done, can you post how the losses (mse*index and mse) varies with epochs and how they affect the models’ performance?
weights are updated only in training phase AFAIK, so it not make sense to use them in validation phase, isnt it?
Yes, but to compare the training loss and validation loss, they must represent the same quantity right? You won’t be able to judge if the model is overfitting or underfitting if the training the validation loss represent different quantities