arch=resnet34
sz=224
bs=64
data=get_data(sz,bs)
learn = ConvLearner.pretrained(arch, data, precompute=True)
learn.lr_find()
learn.sched.plot()
for lr in [0.001, 0.005, 0.01, 0.05, 0.1, 0.2]:
learn = ConvLearner.pretrained(arch, get_data(sz, bs), precompute=True)
print(lr)
learn.fit(lr, 5)
LR find plot
From the plot, it looks like lr=0.1 is suitable. 0.01 is definitely at the beginning of the curve.
but the actual performance seems different.
0.001
epoch trn_loss val_loss accuracy
0 4.769103 4.007295 0.156556
1 3.865272 3.130774 0.436888
2 3.120358 2.483445 0.561644
3 2.602595 2.05493 0.63454
4 2.164704 1.741485 0.687378
0.005
epoch trn_loss val_loss accuracy
0 2.999551 1.802214 0.669765
1 1.619395 1.058552 0.778865
2 1.120092 0.794603 0.801859
3 0.924779 0.694475 0.812622
4 0.78142 0.642564 0.825832
0.01
epoch trn_loss val_loss accuracy
0 2.188612 1.093844 0.764188
1 1.116692 0.731937 0.810176
2 0.821879 0.62099 0.819961
3 0.68206 0.564141 0.829256
4 0.611454 0.546055 0.830235
0.05
epoch trn_loss val_loss accuracy
0 1.249754 0.654354 0.802348
1 0.776606 0.586021 0.815558
2 0.624501 0.608795 0.814579
3 0.506233 0.615481 0.812622
4 0.446815 0.633636 0.810665
HBox(children=(IntProgress(value=0, max=6), HTML(value='')))
0.1
epoch trn_loss val_loss accuracy
0 1.244645 0.711031 0.778865
1 0.818789 0.70887 0.790117
2 0.611264 0.632221 0.817515
3 0.509674 0.658628 0.814579
4 0.438409 0.650543 0.812622
HBox(children=(IntProgress(value=0, max=6), HTML(value='')))
0.2
epoch trn_loss val_loss accuracy
0 1.387966 0.75263 0.77544
1 0.909984 0.744139 0.791096
2 0.77577 0.711475 0.815068
3 0.645953 0.875455 0.787182
4 0.6675 0.776812 0.80137
How can I be choose LR using LR finder with utmost certainty?