Working on integrating a seq-to-seq model training into v2 and would love to be able to pass an ignore_index
that could be used to tell LabelSmoothingCrossEntropy
to not look at certain token ids when calculating the loss (e.g., ignore any -1 token ids) when comparing the generated text to the actual.
In v1, not knowing any better, I just created my own class to create this one modification in the forward
pass:
def forward(self, output, target):
c = output.size()[-1]
output = output[target != self.ignore_index]
target = target[target != self.ignore_index]
log_preds = F.log_softmax(output, dim=-1)
...
Is there a better way in v2?