I am training a binary human segmentation model on human data and my loss/accuracy suddenly job mid-training for no apparent reason. I initlise my model as :
learn = unet_learner(data, models.resnet34, metrics=metrics, wd=1e-2,loss_func=lossf)
and train in two frozen and unfrozen parts. Loss is just plain cross-entropy. The loss jumps in the first part and remains there throughout. Here is how the logs look:
What could be going wrong?