Poor ResNet18 results on digit recogniser

Part 1 (2019)
1

Hello all,

I am getting pretty awful results with resnet18 on digit recogniser.

epoch train_loss valid_loss error_rate time
0 0.759114 0.485745 0.142024 01:02

There is at least one reason for that I can think of. Training data is in gray scale, whereas nn trained for input of 3 channels. The workaround for that is as follows:

training_images_3 = np.concatenate([training_images, training_images, training_images], axis=1)
testing_images_3 = np.concatenate([testing_images, testing_images, testing_images], axis=1)

(42000, 1, 28, 28)
(28000, 1, 28, 28)

(42000, 3, 28, 28)
(28000, 3, 28, 28)

If anyone could have a glance at the notebook I would be grateful for any suggestions. I ran out of ideas

Thanks!

2

train for more epochs

3

I did. After 10 epochs validation loss get as low as 0.1, which is still pretty bad result

4

What’s your error rate?

5

Here the results after 10 epochs

epoch train_loss valid_loss error_rate time
0 1.192827 0.867493 0.267262 01:02
1 0.741926 0.455125 0.138929 01:02
2 0.478559 0.217530 0.060595 01:02
3 0.352874 0.144018 0.038571 01:02
4 0.344563 0.121772 0.032143 01:02
5 0.314101 0.104966 0.028929 01:01
6 0.237036 0.095026 0.024643 01:01
7 0.173674 0.089587 0.023214 01:01
8 0.210568 0.087508 0.022381 01:02
9 0.182282 0.081610 0.020714 01:02
6

so you have 98% accuracy. I don’t see what’s wrong…

7

Hm… probably I missed something while making predictions then. Would you mind having a look at the notebook?

8

Again, a quick glance shows nothing significantly wrong. You trained for one epoch and got 86% accuracy. Train for longer and it will improve obviously.

Only thing I can think of is maybe your plotting code is wrong. But the predictions are probably fine as the confusion matrix looks OK.

Also, I will say your code could be cleaner, and you could probably select the learning rate using the learning rate finder rather than using a default learning rate.

1 Like
9

Appreciate the help. Thanks!

10

No problem, let me know if you have any further problems.

1 Like