How to set shuffle=False of train and val?

fastai
#1

data = ImageItemList
.from_df(df, path, folder=“train”, cols=0, suffix=".png")
.random_split_by_pct(valid_pct=0.2)
.label_from_df(cols=1, sep=" ")
.transform(tras, size=src_size)
.databunch(bs=40, num_workers=40)

How to set shuffle=False of train and val?

Load_learner on text data - None type not subscriptable
#2

The validation dataset always uses shuffle=False. If you want the training dataloader to use shuffle=False, there’s no direct argument but you can type data.train_dl = data.train_dl.new(shuffle=False) after creating the DataBunch.

4 Likes
#3

Hi @sgugger and @Bohdan ,
I am using Databunch.create() using the following lines of code:

ds_train = PyTorchDataset('fld/training/', batch_size = 8)
ds_val = PyTorchDataset('fld/alidation/', batch_size = 8)
from torch.nn.modules.loss import CrossEntropyLoss
model = BaseCNN(n_channels = 1, n_classes = 2)
loss_func = CrossEntropyLoss(weight = torch.Tensor([0.1,0.9]))
data = DataBunch.create(ds_train,ds_val,collate_fn=cnn_collate,path="fld/")

The create function raised the following error:

ValueError: DataLoader with IterableDataset: expected unspecified shuffle option, but got shuffle=True

I don’t know what I am missing out. Can you please help me with that. Thanks in advance.

#4

v1 will not support IterableDataset, only regular datasets with a length and getitem. You can still create your dataloaders by hand then pass them to the init of DataBunch.

#6

Not sure of your error since you only give part of the stack trace but you should try to add fix_dl=train_dl in your call to DataBunch. It’s trying to create it for internal purposes by using our monkey-patch of the PyTorch DataLoader, but that monkey-patch isn’t compatible with your dataloader class.

#8

@sgugger, Just saw this after posting the trace, would try with that

#9

@sgugger,

Thanks it works . Appreciate the help.

#10

Can you guys send the code where you achieved shuffle= true for fastai v1.
And does the shuffling work while training or before the training the imagedatabunch is created by shuffling the data it takes , which is the way , I’m using Fastaiv1.
@sgugger