Validation loss is getting bigger all the time!

Part 1 (2019)
1

Hi Everyone,
I am working on the distracted drivers classification problem using Kaggle states farm dataset. it has 10 classes.
I have started with ResNet-50 with 20% of the data as validation and everything was OK! until I decided to split the validation set by drivers (4 drivers out of the 26) which seems a better idea to match the test set which contains new drivers too.

I started to see strange losses and accuracies for the validation!!

22%20PM
Screen Shot 1440-06-28 at 8.11.22 PM.png964×674 34.1 KB

I tried the following to reduce the overfitting:

  • Data augmentation

  • Dropout in the fully-connected layers [0.5 and 0.7]

  • BatchNorm after the last layer

  • Weight Decay (0.1)

but none of them solve my problem!

Any help would be appreciated :sleepy:

2

Your validation loss starts to increase at the beginning, it means that you overfit right at the first epoch. It makes sense because your model can’t predict on the new classes that they never seen in the training. you should change your approaches, in case of the new drivers in the test set it should classify it as unknown.

3

I am not using new classes they are both have similar classes. but the drivers in the validation set are not seen in the training set (to check if the model will generalize well for not seen data)

4

Would you mind upload your notebook to Github or Kaggle kernel so we can have a look at it? It is pretty hard to tell what is going on just from that screenshot, as there are so many things that could go wrong.

5

sure

note: now i am using only a sample of the data to make my experiments faster.

6

Since it is a high variance problem, I would recommend using the entire dataset and see if it improves the situation first.

7

I tried it before and got same problem
thanks

8

Well, I know that this one sounds like a strange advice, but I would update the fastai library to the latest one and try again.

9

how can I do that While I am using PaperSpace Gradient?

10

Here is a guide

11

thanks for the link. but nothing changed still have the same problem.

12

@jeremy can you help me figure out the problem? Thanks

13

Please read the etiquette section of the FAQ.

15

Thanks so much for your help. Everything is ok now after cleaning the training set.