Things impacting learning rate finder for binary segmentation problems

[bilalUWE (Bilal)]

Hi all,

I’m training a binary segmentation model to identify the background and foreground of a surgical image. The definition of the learner is as follows:

learn = unet_learner(dls, resnet34, self_attention=True, metrics=custom_accuracy).to_fp16()

When I run the learn.lr_find() method, I get the following line shape:

Ideally, the shape should look like the following line where it gradually goes down and then shoots up:

There are 24,000 images in the training set and the batch_size for dls is 32. I think the data volume might not the problem but don’t know what else could impact the determination of a stable lr for binary segmentation. Any ideas for things to further investigate this issue?

I will appreciate any hints.

Many thanks and

Kind Regards,
Bilal

[muellerzr (Zachary Mueller)]

@bilalUWE

Any ideas for things to further investigate this issue?

There isn’t an issue per-se, that’s just how your graph looks. I’ve experienced this myself as well. Typically you can try a bit higher of a learning rate than suggested (so like 1e-3) and see where that goes, but it’s nothing to worry about I believe

[bilalUWE (Bilal)]

@muellerzr

but it’s nothing to worry about I believe

Thanks muellerzr. Glad to hear it’s nothing to worry about. The training on resnet34 is also stable:

And preds vs tags are also reasonably close for most images:

Any idea if the model can be made to learn the rectangular masks?