BCELossFlat

champs.jaideep · January 30, 2020, 12:03pm

Can some one explain what actually is done by FLAT version of BCE loss and how it output works with standard BCE loss functions.
BCE requires last layer output to be same as size of label .

muellerzr · January 30, 2020, 12:11pm

This may help:

github.com

fastai/fastai/blob/master/fastai/layers.py#L249


        input = input.transpose(self.axis,-1).contiguous()
        target = target.transpose(self.axis,-1).contiguous()
        if self.floatify: target = target.float()
        input = input.view(-1,input.shape[-1]) if self.is_2d else input.view(-1)
        return self.func.__call__(input, target.view(-1), **kwargs)


def CrossEntropyFlat(*args, axis:int=-1, **kwargs):
    "Same as `nn.CrossEntropyLoss`, but flattens input and target."
    return FlattenedLoss(nn.CrossEntropyLoss, *args, axis=axis, **kwargs)


def BCEWithLogitsFlat(*args, axis:int=-1, floatify:bool=True, **kwargs):
    "Same as `nn.BCEWithLogitsLoss`, but flattens input and target."
    return FlattenedLoss(nn.BCEWithLogitsLoss, *args, axis=axis, floatify=floatify, is_2d=False, **kwargs)


def BCEFlat(*args, axis:int=-1, floatify:bool=True, **kwargs):
    "Same as `nn.BCELoss`, but flattens input and target."
    return FlattenedLoss(nn.BCELoss, *args, axis=axis, floatify=floatify, is_2d=False, **kwargs)


def MSELossFlat(*args, axis:int=-1, floatify:bool=True, **kwargs):
    "Same as `nn.MSELoss`, but flattens input and target."
    return FlattenedLoss(nn.MSELoss, *args, axis=axis, floatify=floatify, is_2d=False, **kwargs)

We flatten before sending in our input and target

champs.jaideep · January 30, 2020, 12:18pm

Thanks
I checked it out… but was trying to understand how standard bce is able to calculate the loss after this transformation.
in a binary classification i suppose fast ai creates output layer with output size as 2 .
but BCE expects size of label which is one to be same in output .
N * 2 vs N (label)