tmfdDL has two parameters (among others). One is bs=64, the other batch_size=None. Can any one explain to me the difference between the two? How are they meant to be used? A subsequent, show_batch() command crashes the system whenever batch_size is set to anything other than 1.
So it’s just for compatibility. Use
bs. (No idea why setting
batch_size would cause a problem, however).
I have the following code that loads the COCO_TINY dataset:
ds_items = get_image_files(ds_source/‘train’)
ds_split = RandomSplitter(seed=SEED_VL, valid_pct=0.2)(ds_items)
train, valid = (ds_items[i] for i in ds_split)
ds_bbox = lambda o: get_x_y[o.name]
ds_label = lambda o: get_x_y[o.name]
def to_np(x): return np.array(x, dtype=np.float32)
dsets = Datasets(ds_items, [PILImage.create, [ds_bbox, to_np, TensorBBox.create], [ds_label, MultiCategorize(add_na=True)]], splits=ds_split, n_inp=1)
Transformations and dataloaders (WORKS!)
aft_itm = [BBoxLabeler(as_item=False), PointScaler(y_first=False), ToTensor()]
aft_btch = [IntToFloatTensor(), AffineCoordTfm(size=SZ)]
tdl_trn = TfmdDL(dsets, bs=1, num_workers=4, after_item=aft_itm, after_batch=aft_btch, device=default_device())
dldr = DataLoaders(tdl_trn)
If bs is set to 1 in TfmdDL, then the code works fine, tdl_trn.one_batch() runs correctly, and tdl_trn.show_batch( figsize=(5,5)) shows the image. However, if bs=2 (or any number greater than 1), tdl_trn.one_batch() crashes with the following error:
RuntimeError Traceback (most recent call last)
----> 1 x,y,z = tdl_trn.one_batch()
2 type(x), type(y), type(z), x.shape, y
/usr/local/lib/python3.6/dist-packages/torch/utils/data/utils/collate.py in default_collate(batch)
53 storage = elem.storage().new_shared(numel)
54 out = elem.new(storage)
—> 55 return torch.stack(batch, 0, out=out)
56 elif elem_type.module == ‘numpy’ and elem_type.name != 'str’
57 and elem_type.name != 'string’:
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 1 and 2 in dimension 1 at /pytorch/aten/src/TH/generic/THTensor.cpp:689
Your help on how to solve this issue will be appreciated
That’s pretty hard to read. Can you please use markdown code formatting and try to make your post as clear as possible?
My apologies, Sir. I am pretty inept at formatting things. I have edited the code above and hope it is clearer.
coco_tiny images are not all the same size. You should include a Resize method first in your
item_tfms (after_item) to get it working.
That’s the reason I set AffineCoordTfm(size=SZ). At least I thought so.