Loss increases at the end of the cycle


I train a segmentation model (very similar to the camvid notebook ; lesson3-camvid.ipynb ) to segment multiple sclerosis lesions in brain MRI.

The loss decreases, reaches a plateau, then increases for the last few epochs :

What should I do? Retrain the model with fewer epochs?
Is there a rule-of-thumb for this situation?

Here is a few images from my dataset :