Things impacting learning rate finder for binary segmentation problems

Hi all,

I’m training a binary segmentation model to identify the background and foreground of a surgical image. The definition of the learner is as follows:

learn = unet_learner(dls, resnet34, self_attention=True, metrics=custom_accuracy).to_fp16()

When I run the learn.lr_find() method, I get the following line shape:
Screenshot from 2022-07-25 21-51-03

Ideally, the shape should look like the following line where it gradually goes down and then shoots up:
Screenshot from 2022-07-25 21-56-58

There are 24,000 images in the training set and the batch_size for dls is 32. I think the data volume might not the problem but don’t know what else could impact the determination of a stable lr for binary segmentation. Any ideas for things to further investigate this issue?

I will appreciate any hints.

Many thanks and

Kind Regards,

There isn’t an issue per-se, that’s just how your graph looks. I’ve experienced this myself as well. Typically you can try a bit higher of a learning rate than suggested (so like 1e-3) and see where that goes, but it’s nothing to worry about I believe

1 Like

Thanks muellerzr. Glad to hear it’s nothing to worry about. The training on resnet34 is also stable:

And preds vs tags are also reasonably close for most images:
Screenshot from 2022-07-25 22-48-06

Any idea if the model can be made to learn the rectangular masks?