Last training batch is dropped?

It looks like the last batch during training is always dropped (the first True in the tuple below assigned to drop_last param of DataLoader's constructor):

        dls = [DataLoader(d, b, shuffle=s, drop_last=s, num_workers=num_workers, **dl_kwargs) for d,b,s in
               zip(datasets, (bs,val_bs,val_bs,val_bs), (True,False,False,False)) if d is not None]

Is it by design or am I missing something?

1 Like

It is by design, because a lot of models use BatchNorm, which behaves badly if you have a batch of a small size (especially size 1, that will throw an error). To avoid this, we drop the last batch during training (since there is shuffle, it doesn’t have any impact).


If you want the predictions on the training set, you should yse ds_type=DatasetType.Fix which is the training set with shuffle=False and drop_last=False.


Yes thank you :slight_smile: I had deleted my post because after just a little more digging in the forum I found you answered this already in another post. Many sorry for the duplicates :confused: and thank you again :sunny:

1 Like