Tag people with pretrained MaskRCNN on WIDERFace dataset: To do NMS or not to do?

Hello!
I’m going to train a detection model with 2 classes, faces and people. For this task I want to take 2 datasets WIDERFace and WIDERPerson, tag each of it with SotA detectors. I had no problem running face detection on WiderPerson, and now I’m facing a problem with Face dataset.
Consider this 2 images:
On the first one it is clear that we need to use NMS in order to filter out 2 prediction, because it is same person as 0.


For the second image, NMS might be less usefull, because person 1 is overlapping over 5 and 6 in a way 1 will remove 5 and 6 with NMS.

What is the best way to resolve this issue?
I know that there is no correct answer, but what would you pay most attention?
Sincerely yours
Igor