Wow! Awesome resource!
I believe you are overfitting. A good place to stop training would at 10-15 epochs.
Where do you find train_losses and validation_losses?
- Underfitting – Validation and training error high
- Overfitting – Validation error is high, training error low
- Good fit – Validation error low, slightly higher than the training error
- Unknown fit - Validation error low, training error ‘high’
Can someone tell me how to interpret this plot. i wasn’t sure if this is a good fit because the val_loss (test=validation) is smaller than the loss(train).
I got this result with a dropout rate of 0.5, and a learning_rate of 0.001 in an lstm network
1 Like
You should stop it when training it at 10 epos. Do you see the red line is below the blue line? It means your machine is getting good at recognizing trained data but getting worst on unseen data.
1 Like
Hi, what about this result?
here is figure zoom from 1 to 100 epochs:
This thread has been very informative - thanks all!
Based on this information on cross-validation, I am randomly generating hyperparameters to find the best combination of params for my model. For comparing models, is it better to pick the model with the lowest loss? Or better to find a model where the loss and val_loss converge.
For example,
Case 1: Best loss is 0.15 (with the best val_loss being 0.27)
or
Case 2: Best loss is 0.18 and best val_loss is 0.185 (ie they converge)
Which case would be preferred?
Thanks!
I’m going to say you are underfitting
You’re training loss is always worse than your validation loss, and even though it’s bumpy, it trends proportionally higher than the validation loss throughout almost all of the training.
Hello. Can you please tell me if my model is overfitting?
Hey everybody, my validation loss has been consistently lower then my training loss no matter what parameters I change. Because of this, I first thought that the model did not know how to fit, but after doing some inference it turns out it works really well. Although it works as planned, I’m still really curious about what this loss function graph means. Do any of you know what is happening when my model is training, and most importantly what I subsequently can do to improve it?
thanks muchI’m going to respectfully disagree and say this is looking like a good fit. 
Why?
Well notice that while your training loss outperforms you validation loss at around batch 3000, your validation loss is still going down. Overfitting is when your validation loss starts getting worse as your training loss improves … which isn’t the case here. It is improving (just not as much as the training which is expected and desirable).
4 Likes
i think here the model is underfitting. Your training loss always > validation loss. Are you using LR based on learning rate finder ?. Also try increasing the size of your hidden layers. If this a tabular model, have a look at An Attempt to Find the Right Hidden Layer Size for Your Tabular Learner
Can you take your feedback about my result? I read previous comments and posts here so it looks like ok. But I need other comments. Thank you.
I have a problem with my network which is an Unet!
At the very beginning of training, the validation loss starts increasing, and training loss decreases. It is obviously an overfitting problem. I decreased the number of the trainable parameters, used batch normalization and dropout layers, increased L2 regularization parameter, applied different normalization for reference and training datasets, but neither of them did work for me and the result is the same! Do you have any other suggestions?
