Labe smoothing via cnn_learner

Deep Learning
1

Hi.

Is there a way to add loss_func = LabelSmoothingCrossEntropy to a cnn_learner()? This is my code:

learn = cnn_learner(data, models.resnet50, metrics=[accuracy, top_1], callback_fns=ShowGraph)
2

specify it as loss_func=

You also need to pass it in as an instance. IE: LabelSmoothingCrossEntropyLoss()

3

Right but how do I add it to my cnn_learner()?

4

Just pass it in with your call to cnn_learner as a parameter.

5

I did this:

learn = cnn_learner(data, models.resnet50, metrics=[accuracy, top_1], loss_func = LabelSmoothingCrossEntropy, callback_fns=ShowGraph)

And when I ran

learn.lr_find()
learn.recorder.plot(suggestion=True)

I got

/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __getattr__(self, name)
    574                 return modules[name]
    575         raise AttributeError("'{}' object has no attribute '{}'".format(
--> 576             type(self).__name__, name))
    577 
    578     def __setattr__(self, name, value):

AttributeError: 'LabelSmoothingCrossEntropy' object has no attribute 'detach'
6

See what I said in my first comment.

7

@muellerzr following up to this, if say you have two datasets to use to train but you would like to weight on the training samples of one more than the other, how would you go about that?

Say for instance images are in 2 different dataframes so you concatenate them and use ImageItemList.from_df

8

@oo92 @DrC did some digging in my old repos, check out this notebook here. It has the LabelSmoothing code and examples:

Note: even though it may not use cnn_learner, you can always pass in a loss_func

9

That’s a tricky question! One potential would be to ensemble two models with said data, and then weigh ones predictions over the other a certain weight. Such as preds_a = preds_a40%, preds_b60% (for example)

Otherwise unless it’s between classes (which sounding like it’s not), I haven’t looked into that particular area before! :slight_smile:

10

Well, they are not separate since a single batch can have some of dataset 1 and some from dataset 2, but the two have different weights associated with them. Kind of like upweighting imbalanced dataset classes but this is intra-datasets… i put it under this thread because i felt this can be handled with some different loss function inherent in fastai/pytorch!
Any ideas would be great :slight_smile: