Help on KeyPoints Detection Problem - Regression and Classification on a single network


I’m looking to design a network to do keypoints detection on images. These images might be one where the whole face (for example) is visible, so all keypoints are visible, OR where the images are partially cropped, so only a subset of keypoints are visible.

As such, I am planning to have two sets of outputs (y’s): regression (for the keypoints that are affected by the data augmentations - removed if part of the cropped out portion of images), and multi-label classification (where each label tells us if a particular keypoint is visible or not).

Can someone point me to an example I can follow? I know I have to use the ImagePoints class and custom loss functions, but am otherwise stuck.


Hello have you able to solve this problem.