Implement Bag of Tricks for Image Classification with Convolutional Neural Networks

(GENNAN CHEN) #1

Hi! All,

Just finish reading this paper https://arxiv.org/abs/1812.01187
Wondering if anyone have tried to implement those tricks in fastai/pytorch? especially some of training tricks like label smoothing, Knowledge distillation and Mixup Augmentation. If not, I am actually interested in giving it a shot.

Gen

0 Likes

(Jeremy Howard (Admin)) #2

Yes they’re pretty much all implemented in examples/train_imagenet_adv.py in fastai repo :slight_smile:

3 Likes

(Shai Alon) #3

Adding Some links for future searchers:
Label Smoothing: loss_func = LabelSmoothingCrossEntropy()

Mixup Augmentation: .mixup(alpha=0.2)

As for Knowledge distillation: I am not sure the imagenette -> imagewoof does this. Would appreciate an explainer if it’s actually implemented.

2 Likes

(Jeremy Howard (Admin)) #4

It’s not. I don’t feel like that really counts as model training - it’s more like model pruning.

1 Like

(Sairam) #5

Jeremy, would the implementation of the LabelSmoothingCrossEntropy() be directly usable for multi-class multi-label classification ?

1 Like

#6

Nope…

0 Likes