Last training batch is dropped?

It looks like the last batch during training is always dropped (the first True in the tuple below assigned to drop_last param of DataLoader's constructor):

        dls = [DataLoader(d, b, shuffle=s, drop_last=s, num_workers=num_workers, **dl_kwargs) for d,b,s in
               zip(datasets, (bs,val_bs,val_bs,val_bs), (True,False,False,False)) if d is not None]

Is it by design or am I missing something?

1 Like

It is by design, because a lot of models use BatchNorm, which behaves badly if you have a batch of a small size (especially size 1, that will throw an error). To avoid this, we drop the last batch during training (since there is shuffle, it doesn’t have any impact).

4 Likes

If you want the predictions on the training set, you should yse ds_type=DatasetType.Fix which is the training set with shuffle=False and drop_last=False.

5 Likes

Yes thank you :slight_smile: I had deleted my post because after just a little more digging in the forum I found you answered this already in another post. Many sorry for the duplicates :confused: and thank you again :sunny:

1 Like