Which loss function is used by the fast ai multi-label multi-classification lesson?
nn.BCEWithLogitsLoss or
nn.MultiLabelMarginLoss
If 2, then it means we can’t use class weights to handle class imbalance problem. Has anyone used nn.BCEWithLogitsLoss with class weights to handle the class imbalance problem in multilabel classification? If so, I would like to know if it worked for you.
I have tried the following but I get dimension errors.
Hi @shbkan!
It looks like your model’s last layer is generating bigger tensor than it is supposed to generate.
if you have made your own model by nn.Sequential(…), then check the size of tensor it is generating at the last layer.
I don’t think the error is due to that. Error only comes when I set the loss function to this customized version. Its something with the way the custom function is set to learn.loss_func.func
Then I think this loss function is flating out the last tensor but not flating the label tensor.
I have faced this error and I am pretty sure that this error is due to the mismatch between the tensors.
I have not figured it out yet. I tried a few different things but could not make it work. I hope someone explain using customized loss function with fastai in general because I see a few posts where people are struggling with it. It is an important topic as well because most of the real-life datasets are not balanced so we need to use weight-sensitive loss functions.
I am sure you will see a length of 576. It seems that class_counts does not only store single classes but also bundles of possible classes or different combinations of multi-label classes. You can check that with np.unique(data.y.items) which will show you results looking like list([41, 43, 44]). Hence you have to use another way of generating your weights. In my case I only had one such class with one observation, so I simply excluded it, I am sorry that I can’t share any more useful code with you.
On a side note I think you need to change weights = np.sum(class_counts)/class_counts to weights=class_counts/np.sum(class_counts) for the correct weight.
You could use nn.BCEWithLogitsLoss to train your model; the only downside is Learner.predict will give you raw outputs (without the sigmoid activation and thresholding).
The issue is caused by BaseLoss's (the parent class that all fastai loss functions subclass) initialiser having flatten=True as the default option and not being exposed in BCEWithLogitsLossFlat's initialiser.