Mixup error

Part 1 (2019)
1

I want to use mixup for a multi-class image classification problem and simply added .mixup() to my already working learner, but got the following error:

RuntimeError Traceback (most recent call last)
in ()
1 learn = cnn_learner(data,models.resnet18).mixup()
----> 2 learn.lr_find()

/opt/conda/lib/python3.6/site-packages/fastai/train.py in lr_find(learn, start_lr, end_lr, num_it, stop_div, wd)
30 cb = LRFinder(learn, start_lr, end_lr, num_it, stop_div)
31 epochs = int(np.ceil(num_it/len(learn.data.train_dl)))
—> 32 learn.fit(epochs, start_lr, callbacks=[cb], wd=wd)
33
34 def to_fp16(learn:Learner, loss_scale:float=None, max_noskip:int=1000, dynamic:bool=True, clip:float=None,

/opt/conda/lib/python3.6/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
194 callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
195 if defaults.extra_callbacks is not None: callbacks += defaults.extra_callbacks
–> 196 fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
197
198 def create_opt(self, lr:Floats, wd:Floats=0.)->None:

/opt/conda/lib/python3.6/site-packages/fastai/basic_train.py in fit(epochs, learn, callbacks, metrics)
97 cb_handler.on_epoch_begin()
98 for xb,yb in progress_bar(learn.data.train_dl, parent=pbar):
—> 99 xb, yb = cb_handler.on_batch_begin(xb, yb)
100 loss = loss_batch(learn.model, xb, yb, learn.loss_func, learn.opt, cb_handler)
101 if cb_handler.on_batch_end(loss): break

/opt/conda/lib/python3.6/site-packages/fastai/callback.py in on_batch_begin(self, xb, yb, train)
276 self.state_dict.update(dict(last_input=xb, last_target=yb, train=train,
277 stop_epoch=False, skip_step=False, skip_zero=False, skip_bwd=False))
–> 278 self(‘batch_begin’, mets = not self.state_dict[‘train’])
279 return self.state_dict[‘last_input’], self.state_dict[‘last_target’]
280

/opt/conda/lib/python3.6/site-packages/fastai/callback.py in call(self, cb_name, call_mets, **kwargs)
248 if call_mets:
249 for met in self.metrics: self._call_and_update(met, cb_name, **kwargs)
–> 250 for cb in self.callbacks: self._call_and_update(cb, cb_name, **kwargs)
251
252 def set_dl(self, dl:DataLoader):

/opt/conda/lib/python3.6/site-packages/fastai/callback.py in _call_and_update(self, cb, cb_name, **kwargs)
238 def call_and_update(self, cb, cb_name, **kwargs)->None:
239 “Call cb_name on cb and update the inner state.”
–> 240 new = ifnone(getattr(cb, f’on{cb_name}’)(**self.state_dict, **kwargs), dict())
241 for k,v in new.items():
242 if k not in self.state_dict:

/opt/conda/lib/python3.6/site-packages/fastai/callbacks/mixup.py in on_batch_begin(self, last_input, last_target, train, **kwargs)
26 new_input = (last_input * lambd.view(lambd.size(0),1,1,1) + x1 * (1-lambd).view(lambd.size(0),1,1,1))
27 if self.stack_y:
—> 28 new_target = torch.cat([last_target[:,None].float(), y1[:,None].float(), lambd[:,None].float()], 1)
29 else:
30 if len(last_target.shape) == 2:

RuntimeError: invalid argument 0: Tensors must have same number of dimensions: got 2 and 3 at /opt/conda/conda-bld/pytorch_1549630534704/work/aten/src/THC/generic/THCTensorMath.cu:74

Is there some bug or am I doing something wrong?

1 Like
2

@sgugger Hello sorry to bother you but do you know anything about this error? I tried looking into the source code to see if it could shed some light into the issue but it didn’t really help.

3

I think the problem is your data. It looks like you put 2D array without channel as your image data. Try this code.

x,y = data.one_batch()
print(x.shape)

You should get 4 dimensions (1 for number of batches), 1 for channels, 2 for size of the image.

torch.Size([64, 3, 224, 224])
4

That doesn’t seem to be the problem. I get:

torch.Size([64, 3, 128, 128])
5

Without information what data is I cannot say anything more :frowning: . Can you give us example?

6

For what it’s worth I get the same error when I try to use mixup on a multi-class problem.

1 Like
7

@klemenka I just took this kaggle kernel and added mixup
@JoshVarty I am also doing a multi-class classification problem. But that should be an issue since it was tested on MNIST, a multi-class problem as well.

8

MNIST is not multi-class as each picture represents exactly one number. In the contest you’ve linked each audio clips can contain multiple sounds.

1 Like
9

Indeed you are right, but the correct term is multi-label not multi-class. MNIST is a multi-class (one label, multiple classes) but ours are multi-label (multiple labels, multiple classes) problems. Mixup seems like it can only be used for multi-class problems (like MNIST) rather than multi-label problems.

10

I think that the problem is that mixup is not for multi classification problem. This implementation needs one classification from first image, and second classification from the second image and mix it up. If you got multiclass than you’ve got an error on the line:
In lambd, the mixup store only values for one classification, but you have only for one. You can try with the mixup with the parameter mixup(stack_y=False),

new_target = torch.cat([last_target[:,None].float(), 
                        y1[:,None].float(), 
                        lambd[:,None].float()], 1)

but I’ve got error: FlattenedLoss' object has no attribute 'get_old'. From me, try to do single classification or put in the input only the strongest classification.

I think that this function was prepared only for single output, and I don’t know how it might work for multiple output. Mixup function based on the source documentation mix two classes. If you have more than one assigned to your image than how you want to mix with another one?

1 Like
11

That shouldnt be the case. The fastai docs on mixup said it was used successfully on cifar10

12

But CIFAR10, and 100 has single class images, not multiple.

image

#CIFAR10 example

from fastai import *
from fastai.vision import *

path = untar_data(URLs.CIFAR)


data = (ImageList
        .from_folder((path/'train'))
        .split_by_rand_pct(valid_pct=0.2)
        .label_from_folder()
	     .transform(tfms=get_transforms(), 
                  size=100)
       .databunch(bs=16)
       .normalize(imagenet_stats))

learn = cnn_learner(data, models.resnet34, metrics=error_rate).mixup()
learn.fit_one_cycle(1,max_lr=0.2)

image

13

There is a difference between multi-class classification and multi-label classification. CIFAR10 and MNIST are all multi-class but not multi-label. Mixup (as currently implemented) does not work with multi-label problems.

14

This bug has been fixed in master, if you make a dev install, you shouldn’t see it anymore.

1 Like
16

Will there be an implementation of mixup for multilabel problems? I found one paper that adapted mixup for multilevel problems but I wonder how effective it is.

17

Mixup already is implemented for multilabel problems. There was a bug for it which has been fixed in master and will be in the next release, that’s what I was pointing out in my previous comment.

18

Ah okay thanks! Approximately when will the next release be?

19

In a couple of hours actually :wink:

1 Like
20

Unfortunately, this did not solve the original error I received when using mixup for my multi-label problem. I don’t know if it is something I am doing wrong or if the library still does no support multi-label problems and that there is a bug in the mixup code. I confirmed using fastai.__version__ that I am using fastai version 1.0.52

21

Did you pass stack_y=False to mixup? Also don’t just post the end of your error message, but the code you’re running and the full stack trace, it makes helping you easier for others.

2 Likes