I have observed in training of datasets in Lesson 3 (like Camvid) that validation loss is generally lower than training loss in each epoch.
Doesn’t that mean my model has high bias? I expect training loss to be smaller as model is fitted to that data more closely.
60.00% [6/10 09:28<06:18]
| epoch | train_loss | valid_loss | acc_camvid |
|---|---|---|---|
| 1 | 1.007837 | 0.906481 | 0.777404 |
| 2 | 0.745094 | 0.613785 | 0.846059 |
| 3 | 0.677771 | 0.646452 | 0.835799 |
| 4 | 0.619955 | 0.538954 | 0.848037 |
| 5 | 0.603284 | 0.462265 | 0.877266 |
Yes, you can train for longer and the accuracy should keep improving. When the accuracy starts to decrease then it will be a sign of overfitting, until then you can keep training. I believe the validation loss is lower because during the validation phase dropout is not active.
I don’t think the validation loss is lower because during the validation phase dropout is not active.
At 51m53s of the lesson 2 video, ps = 0 means turn off the dropout, but train_loss is still higher than valid_loss.
I’m having the same problem with the MNIST dataset , using default LR, trying differents arquitetures (resnet18, resnet50), image size (16,32,64), splinting the validation with more data (0.3), but everytime I’m getting the training loss bigger than the validation:
am I doing something wrong?
mnist = untar_data(URLs.MNIST)
tfms = get_transforms(do_flip=False)data = (ImageItemList.from_folder(mnist)
.random_split_by_pct(0.3)
.label_from_folder()
.transform(tfms, size=32)
.databunch()
.normalize())learn.fit_one_cycle(3, slice(1e-5))
Total time: 05:34
| epoch | train_loss | valid_loss | accuracy |
|---|---|---|---|
| 1 | 0.062921 | 0.029518 | 0.990286 |
| 2 | 0.064949 | 0.027088 | 0.991333 |
| 3 | 0.058849 | 0.026884 | 0.991238 |
| learn.fit_one_cycle(3, slice(1e-5)) |
I find that it was needed to use regularization to solve that, so using it
learn = create_cnn(data, models.resnet18, metrics=accuracy, ps=0, wd=0)
the result was:
40 0.014386 0.082150 0.994571 01:59
I update the jupyter. more suggestions are welcome.
