Lesson 2 In-Class Discussion ✅

[dhruvt93 (Dhruv Thakur)]

I’m getting the following plot after running learn.lr_find(). I’ve chosen the lr range that corresponds to the middle drop. I can see that the valid_loss is beginning to increase on the 6th epoch, but what can be the reason for a constant error_rate? Let me know if more context is required.

[dhruvt93 (Dhruv Thakur)]

P.S. This is the case when lr is in range (1e-6,4e-6) too.

[aloksaan (Alok)]

Updating the fastai library fixed it

[marcmuc (Marc P. Rostock)]

answered here

Why does "train-loss > val-loss" mean low learning rate?

From what I understand it’s the following. The model trains on the training set, and the training loss is calculated on that training set. So this loss is calculated on the same dataset the model was trained on and should thus be low. However, the model has never seen the validation set, so when the validation loss is calculated it is on new data for a model only trained on the training set. Hence the training loss ought to be lower than the validation set, because the model was fitted on the…

[MaheshKhatri]

Hope this helps

https://machinelearningmastery.com/tactics-to-combat-imbalanced-classes-in-your-machine-learning-dataset/

[MaheshKhatri]

Yes. All non apple and non orange images in training set to be classified as Other.

[sgugger]

No it’s not a random direction, but the direction of the gradients. The stochastic refers to the fact we draw batches randomly.

[MaheshKhatri]

Rename then with the most correct labels from a human perspective.

[MaheshKhatri]

Michael, Have you seen this ?

Rescaling an image when resolution does not meet size criteria (e.g 224)

Again, been following the code to see how a specific problem is solved: i.e. what do you do with images when the input size is not the exact prescribed dimensions (e.g. 224 x 224, 229 x 229 etc) I think this code is at the kernal of resizing: def scale_min(im, targ): """ Scales the image so that the smallest axis is of size targ. Arguments: im (array): image targ (int): target size """ r,c = im.size ratio = targ/min(r,c) sz = (scale_to(r, ratio, targ), …

[MaheshKhatri]

As Andrew Ng has said, it is like worrying about over population on Mars.

[jeremy (Jeremy Howard)]

I found another paper that shows how a variational autoencoder can be used to generate synthetic examples from the minority class, similar to the SMOTE and ADASYN techniques used in traditional ML.

I removed that one since it’s not the approach we recommend.

[jeremy (Jeremy Howard)]

What’s happening here? Everytime learn.fit_once_cycle(4) is called, expectance of particular size of tensor is changing everytime. Moreover Isn’t create_cnn supposed to handle this from data ? I’m on v1.0.18 @sgugger

Please read the etiquette guide in the FAQ.

[joshfp (José Fernández Portal)]

I’m facing the same issue and agree that replacing valid_ds with train_ds is not ok. Looking at the code of the ClassificationInterpretation class, it seems that it only works with the validation set. I guess would be great that it would receive a parameter to select the dataset.

[MaheshKhatri]

This may help.

https://arxiv.org/abs/1809.01442

[jeremy (Jeremy Howard)]

Question: Image size, size parameter in ImageDataBunch, that is set to 224. Is it better that is you will get lower error when images are higher resolution (it will take more time, bs smaller)? or the image resolution need to be 224 for resnet34?

You will get lower error when using higher res images.

[poppingtonic (Brian Muhia)]

That’s OK. Autoencoders are too noisy anyway, which means the NN learns the quirks of the autoencoder, rather than what really makes an image a member of a class.

[MaheshKhatri]

In the video, Jeremy has explained this very clearly

[vedder]

Thanks I’ll be sure to check it out. I am using the same dataset so this could be really helpful!

[MaheshKhatri]

@miwojc has kindly shared this Jeremy paper on another forum. It may help.

1802.01528.pdf

[MaheshKhatri]

Sorry. I did not see that you had already shared the same resource. Thanks.