PyTorch Multi Output 50%

Using torch.legacy.nn we can use ParallelCriterion to do multi-output with PyTorch–yay!

However, we also need to use a custom dataset to feed that multi-output in. How do we do that? I’ve currently followed this tutorial to create a custom dataset, which works to my likings. It returns X, (label, regression) where label is a standard, one-hot encoded torch inttensor, and regression is an int torch tensor as well for my bounding box x,y,w,h.

After instantiating my dataset, when I drop it into torch.utils.data.DataLoader(), pytorch complains that:

can’t convert a given np.ndarray to a tensor - it has an invalid type. The only supported types are: double, float, int64, int32, and uint8.

I’m assuming it does not like the tuple (label, regression) and is expecting a regular torchtensor label instead. Any suggestions… =) ?

Can you link to your code? Or paste it here with the python styling.

Was able to get it working. Here’s the take home:

Custom dataset can return a tuple of as many torch tensors as possible. Don’t combine them like keras into nested tuples, just make all the returned values separate tensors, e.g. return img, img_class, img_bbox, ..., whatever_else_you_want_here.

Two: you can combine your losses and weigh them however you want:

loss_ce  = nn.CrossEntropyLoss(weight=None)
loss_mse = nn.MSELoss()

for images, labels_targets, labels_bboxes in train_loader:
    
    images = Variable(images).cuda() 
    labels_bboxes  = Variable(labels_bboxes).cuda()
    labels_targets = Variable(labels_targets.view(-1)).cuda()
    
    optimizer.zero_grad()
    ctypes, bboxes = model(images)
    
    lossA = loss_mse(bboxes, labels_bboxes)
    lossB = loss_ce(ctypes, labels_targets)
    
    # Damn this is hot:
    loss = lossA * 0.001 + lossB * 1.000
    loss.backward()
    optimizer.step()

Takehome #3, read the manual. Pytorch’s CrossEntropyLoss doesn’t expect a 1hot-encoded target, lol.