Multiple object detection with ItemPoints

I am trying to setup a net to detect multiple bright objects on a dark background, but i am having problems when there is an uneven number on each picture.

I have setup the code similar to the head pose notebook from lesson 3, and i padded my Image points list to have even number of elements.

def multipoint_pad_collate(samples:BatchSamples) -> Tuple[FloatTensor, Tuple[FloatTensor]]:
max_len = max([len(s[1].data) for s in samples])
coordinates=torch.zeros(len(samples), max_len, 2)
imgs = []
for i,s in enumerate(samples):
imgs.append(s[0].data[None])
coord = s[1].data
coordinates[i,-len(coord):]=coord

return torch.cat(imgs,0), (coordinates)

However when running the code with uneven number of image points i get the error:
RuntimeError: The size of tensor a (128) must match the size of tensor b (208) at non-singleton dimension 0

Any idea if it is possible to run with uneven number of image points, because the way it ignores the padding makes me wonder if it is?