Thank you so much for your reply.
I copy-pasted the PyTorch code and called it in learner. However i am having a doubt about what do I pass in activation when I call this in Learner.
Below is the PyTorch code from bi-tempered-loss-pytorch/bi_tempered_loss_pytorch.py at master · fhopfmueller/bi-tempered-loss-pytorch · GitHub.
def bi_tempered_logistic_loss(activations, labels, t1, t2, label_smoothing=0.0, num_iters=5):
"""Bi-Tempered Logistic Loss with custom gradient.
activations: A multi-dimensional tensor with last dimension `num_classes`.
labels: A tensor with shape and dtype as activations.
t1: Temperature 1 (< 1.0 for boundedness).
t2: Temperature 2 (> 1.0 for tail heaviness, < 1.0 for finite support).
label_smoothing: Label smoothing parameter between [0, 1).
num_iters: Number of iterations to run the method.
A loss tensor.
if label_smoothing > 0.0:
num_classes = labels.shape[-1]
labels = (1 - num_classes / (num_classes - 1) * label_smoothing) * labels + label_smoothing / (num_classes - 1)
probabilities = tempered_softmax(activations, t2, num_iters)
temp1 = (log_t(labels + 1e-10, t1) - log_t(probabilities, t1)) * labels
temp2 = (1 / (2 - t1)) * (torch.pow(labels, 2 - t1) - torch.pow(probabilities, 2 - t1))
loss_values = temp1 - temp2
return torch.sum(loss_values, dim=-1)