If we are trying to tackle some new problems that has not been attempted, how should we go about selecting/devising our loss function?
Generally, based on domain expertise - you need to think about what defines a “good answer” in your domain. Then you need to come up with something that gives that answer, and is differentiable.