Lesson 2 In-Class Discussion

Part 1 (2018)
23

Thanks!

24

How come the validation loss is smaller than the training loss?

5 Likes
25

Is lesson1 using the test data at all? I don’t see a cats/dogs split in the test data dir.

26

Why aren’t we looking at validation accuracy? I thought validation accuracy and validation loss don’t correspond linearly. Or was it validation error?

27

In Kaggle competitions you don’t get the labels for the test dataset. You can only predict over the test dataset.

28

There’s a recent paper called “Don’t Decay the Learning Rate, Increase the Batch Size” Is adjusting the learning rate the most effective for converging or is adjusting the batch size effective as well?

2 Likes
29

Test data doesn’t have labels. These are unlabeled images from the Kaggle competition. If you want to make a Kaggle submission, you should get these images, predict a class using your model and upload predictions to Kaggle.

30

So the labels weren’t released after the fact?

31

As far as I know, correct labels for the test set are not available at kaggle.com.

1 Like
32

You have to have super GPUs to increase enough batch size

6 Likes
34

what’s the atom optimizer?

35

https://arxiv.org/abs/1506.01186

36

Cyclical Learning Rate

1 Like
37

–

1 Like
38

Adam, he will explain it later.

1 Like
39

Actually, it is Adam. Another optimizing method

40

Does this Cyclical Learning rate method work for all ML architectures, or is it meant for this sort of classification problem?

2 Likes
41

Does the learning rate finder work with all types of NNs or just CNNs?

1 Like
42

For all.

43

I didn’t get the reason for not picking the minima of the learning rate vs loss curve, but a rate slightly higher.

3 Likes