Hi,
Is there is a ‘beginner/intermediate’ friendly way to do oversampling in fastai2?
I’ve seen this post on this forum: [Oversampling Callback]. But it’s only for fastai1 I think. Getting this to work on fastai2 would be very useful, since i’m interested in medical imaging and most medical image datasets are highly unbalanced.
We do not yet, but if you’re up to it a PR would be welcome 
(We just have a weighted DataLoader)
If the class is indicated in the DataFrame as category_id, then this should work:
from collections import Counter
count = Counter(df.category_id).values()
class_weights = 1/np.array(list(count))
wgts = class_weights[dsets.train.items['category_id']]
dls = dsets.weighted_dataloaders(path=path,bs=bs,after_batch=batch_tfms,wgts=wgts)
I guess the only thing to note is that you need to create a Dataset instead of directly creating DataLoaders.
The last time I tried to use the weighted dataloader I could not get it working properly, so I used a different methodology: exporting pandas DataFrames, performing the oversampling there then converting it back. It’s easier than it sounds, and the full example is here
@muellerzr I don’t think i’m at that level yet, but might be a nice project for the future.
Is there an advantage of a weighted DataLoader above oversampling?
@ilovescience Thanks, this seems like a nice solution. I’m going to try it.
I am not sure about it, but I think using the weighted dataloaders is more memory efficient than a classic oversampler that replicates data, just because the dataloder passes the items from the minority class more frequently to the model, but without an explicit replication…(I should double check this though)
Anyway, what I found to work better in my problems with imbalanced data is just to assign weights in the loss function, so that you penalize more the errors in the minority class.
wanted to double check, the way way you calculated the weights?
Is it the same as how you would calculate the weights for a weighted dataloader? ie 1/number_of_samples_per_class or were there any tweaks ?
I have very samples of certain classes so my ratio is 2000 to 1 and i was not getting great result with weighted loss functions.
I usually apply the rule 1/n_samples_in_class in the weighted loss function and it works well in terms of Recall and F1, which are the metrics I am interested in in this type of problems.
Is it possible to fix the typo in the thread title?
Very useful thread
Hmmm…
I’m getting this error when I try this:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-14-eb2169037660> in <module>()
2 count = Counter(df.label).values()
3 class_weights = 1/np.array(list(count))
----> 4 wgts = class_weights[dsets.train.items['label']]
5 dls = dsets.weighted_dataloaders(path=path,bs=bs,after_batch=batch_tfms,wgts=wgts)
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
Are we trying to multiply each sample in a class by the class weight?
This does not answer your question.
But providing an example for folks who want to apply Weighted Cross-Entropy as the loss function for the imbalanced dataset.
#Get weights based on the class distribution in the training data
def get_weights(dls):
# 0th index would provide the vocab from text
# 1st index would provide the vocab from classes
classes = dls.vocab[1]
#Get label ids from the dataset using map
#train_lb_ids = L(map(lambda x: x[1], dls.train_ds))
# Get the actual labels from the label_ids & the vocab
#train_lbls = L(map(lambda x: classes[x], train_lb_ids))
#Combine the above into a single
train_lbls = L(map(lambda x: classes[x[1]], dls.train_ds))
label_counter = Counter(train_lbls)
n_most_common_class = max(label_counter.values());
print(f'Occurrences of the most common class {n_most_common_class}')
# Source: https://discuss.pytorch.org/t/what-is-the-weight-values-mean-in-torch-nn-crossentropyloss/11455/9
weights = [n_most_common_class/v for k, v in label_counter.items() if v > 0]; return weights
#Get the weights from classification dataloader
weights = get_weights(dls_cls)
class_weights = torch.FloatTensor(weights).to(dls_cls.device)
learn_cls.loss_func = partial(F.cross_entropy, weight=class_weights)
Any updates on this? I tried the WeightedDL but couldn’t make it work.
I could train with the method @msivanes provided, but when I exported the learner and loaded it again it caused some issues with prediction.
The solution I found was to change
learn_cls.loss_func = partial(F.cross_entropy, weight=class_weights)
to
learn_cls.loss_func = CrossEntropyLossFlat(weight=class_weights)
I missed updating the answer. @muellerzr assisted me with this issue. No need to use pytorch. What you did is exactly the right Solution.
Reason