Use CategoryList for binary classification，but the sum of preds !=1?

Part 2 (2019)
1

hi,every body, now i meet with a problem in the learn.get_preds. i can’t get the sum of preds’ probabilities equal to 1.

The situation is this: i try to custom a loss function called focal loss to tabular learner:

image
image.png1582×1264 192 KB

and the dataloader is like this

image
image.png2336×482 118 KB

the learner :

image
image.png1866×304 42.2 KB

the trained result is fine:

image
image.png938×496 43.6 KB

but I can’t make the sum of the predicted probabilities equal to one !

it show that :

image
image.png1384×554 53.3 KB

can any one help me ?? i think there should be a sigmoid like i use the BCE LOSS,and It automatically helps me normalize the probability

i have looking for the answer through the forum, like this :


image
image.png1566×400 59.7 KB


image
image.png1600×718 113 KB

2

If you look at the definition of get_preds in the basic_train.py file: https://github.com/fastai/fastai/blob/master/fastai/basic_train.py#L333, you’ll see that the choice of a final activation function is made by either using the function you pass as the parameter activ or by using the helper function _loss_func2activ. The helper function doesn’t have a mapping from your focal loss to an activation function and so noop or the identity function is returned.

activ = ifnone(activ, _loss_func2activ(self.loss_func))

def _loss_func2activ(loss_func):
    if getattr(loss_func,'keywords',None):
        if not loss_func.keywords.get('log_input', True): return
    axis = getattr(loss_func, 'axis', -1)
    # flattened loss
    loss_func = getattr(loss_func, 'func', loss_func)
    # could have a partial inside flattened loss! Duplicate on purpose.
    loss_func = getattr(loss_func, 'func', loss_func)
    cls_name = camel2snake(loss_func.__class__.__name__)
    if cls_name == 'mix_up_loss':
        loss_func = loss_func.crit
        cls_name = camel2snake(loss_func.__class__.__name__)
    if cls_name in loss_func_name2activ:
        if cls_name == 'poisson_nll_loss' and (not getattr(loss_func, 'log_input', True)): return
        return _loss_func_name2activ(cls_name, axis)
    if getattr(loss_func,'__name__','') in loss_func_name2activ:
        return _loss_func_name2activ(loss_func.__name__, axis)
    return noop

Therefore when you call get_preds you’re getting the raw activations from the final layer of your network.

To rectify this you could either pass the parameter activ a softmax function or simply apply the softmax function yourself to the result of get_preds.

2 Likes
3

thank you sir!
as far as i see, A PR has been post to the github to deal with this situation,thank you for your thanksgiving~
ps, the pr’s motivation from this url

4

hi, thanks for your kindly reply!!!

i read 5 times to your reply, i also have the final question:

if i have to add a sofmax func to the get_preds like probabilities , when the model is trained, the return valid focal loss and metric are right as i using the default loss and metric?

because when i change the loss from BCE to Focal loss ,the valid F1 score from 0.4 to 0.6, it’s a big
advance ！ …i somehow doubt the result …

waiting for your reply, thank you again !!!

5

Hi,

You need to compare the definitions of the two losses to see what inputs they expect.

Your focal loss expects the activations from the last layer as inputs (before any softmax function). You can see this because F.log_softmax(input, dim=1) is called on the inputs in the forward function of your loss.

Fastai uses the pytorch the flattened version of CrossEntropyLoss by default - see the docs here: https://pytorch.org/docs/stable/nn.html#torch.nn.CrossEntropyLoss.

This criterion combines nn.LogSoftmax() and nn.NLLLoss() in one single class.

So as far as I understand, CrossEntropyLoss also expects activations from the final layer.

Good luck!