Working on integrating a seq-to-seq model training into v2 and would love to be able to pass an
ignore_index that could be used to tell
LabelSmoothingCrossEntropy to not look at certain token ids when calculating the loss (e.g., ignore any -1 token ids) when comparing the generated text to the actual.
In v1, not knowing any better, I just created my own class to create this one modification in the
def forward(self, output, target): c = output.size()[-1] output = output[target != self.ignore_index] target = target[target != self.ignore_index] log_preds = F.log_softmax(output, dim=-1) ...
Is there a better way in v2?