I’d often find running fit_one_cycle over multiple epochs, the accuracy and valid_error would start off terrible (worse than the end of the previous call of fit_one_cycle!) in the first epoch and get better over subsequent epochs. That makes me think they’re not the same.
Sorry typo. I mean the other way around.
I will ask the 3 highly voted questions when Jeremy finishes this explanation
Why do we need to fit_one_cycle with a certain learning rate before unfreezing. Why don’t we instead unfreeze directly.
11 Likes
For human assigned labels, what if human may have different criteria for assigning labels. as a result the labels may be “mislabeled” (but not necessarily wrong,could be different opinions), which may confuse the model; which then affect the model accuracy. What to do with those cases if i dont want to delete them?
Being able to correct incorrect labels is a feature we plan to add in the future
8 Likes
“Matrix multiplication” = “dot product”?
I think the explanation for this is that the optimizer resets, and it’s using momentum, so by the end of the previous training it had a better “idea” of which direction to head, whereas when you restart it starts off by heading in a less optimal direction.
3 Likes
No, absolutely not.
The cycle of one cycle follows this structure. The cycle length of the total move up and down is accros the number of epochs. So if you do one epoch and then run it again, it will do the “triangle” twice. @edwardjross, this also explains why the learning rate at the beginning is higher than at the end of the previous cycle!
Sorry, @sandmann, answered somehow to the wrong question, this was in answer to you.
4 Likes
Also, for metrics, instead of error rate can use F1 score instead of error rate… as @wdhorton mentioned, sampling should be done, based on the class distribution
Never thought like that simple equation y = mx + c can be written as matrix dot product.
2 Likes
not only the momentum, but also the LR itself!
Sorry I meant underfitting I just made a mistake about the losses. Sorry, morning with no coffee over here.
Question: Image size, size parameter in ImageDataBunch, that is set to 224. Is it better that is you will get lower error when images are higher resolution (it will take more time, bs smaller)? or the image resolution need to be 224 for resnet34?
2 Likes
is there a no.of.epochs learner like lr learner?
2 Likes
For imbalanced classes question: would you balance you validation set or check on unbalanced validation set?
3 Likes
Jeremy’s answer on handling unbalanced data eg 200 real bears and 50 teddy bears “just try it, it always just works fine”… is that only when starting with a well-trained net, or is also when starting fresh?
2 Likes
If consciousness arrises from complex enough data processing, at what point do you give up on ML training and obsess over AI sentience?
2 Likes
The model isn’t evenly trained. There’s the resnet backbone, which has been extensively trained on all of Imagenet, and the head, which we add on for our classification purpose and is entirely untrained.
If you trained the entire model at once, you could get large errors coming from the untrained layers back propagating through the model and messing up your nicely pretrained weights.
Training with the backbone frozen allows us to only trained the untrained layers in the head. Once those layers have converged somewhat, we unfreeze the entire model and continue training.
12 Likes
Its OK. I just saw what you meant. It was good question.