How does `split_by_idxs` work?

I wanted to do something custom with my dataset regarding the splitting into training and validation set, and looked into the split_by_idxs functionality. How does this function work? If I have a dataframe which I pass into ImageItemList, do I just give a list of indices of the dataframe that correspond to the training and validation dataset? Are there any examples of this? There doesn’t seem to be much documentation on this section…

That sounds about right. If the indices 100:1000 were for the validation set and the rest for the training set, you could do something like:

data = (ImageItemList.from_df(df)

Otherwise you can use split_by_idxs and set both the training and the validation indices instead of just the validation.


Not really adding much to Tom2718’s explanation but adding the relevant referenced code sections below and adaptations of Tom’s example. Tom, wouldn’t you have to specify valid_idx or train_idx for your example before range(100,1000), see my example below? I think you could pass either train or valid in a range or an interrupted list of ranges:

data = (ImageItemList.from_df(df)


data = (ImageItemList.from_df(df)


def split_by_idxs(self, train_idx, valid_idx):
    "Split the data between `train_idx` and `valid_idx`."
    return self.split_by_list(self[train_idx], self[valid_idx])


def split_by_list(self, train, valid):
    "Split the data between `train` and `valid`."
    return self._split(self.path, train, valid)

.split_by_idx is different than .split_by_idxs and .split_by_idx only requires specification of the validation set indices it seems:

split_by_idx [source]
split_by_idx ( valid_idx : Collection [ int ]) → ItemLists
Split the data according to the indexes in valid_idx .


@ilovescience how we can do this in fastai2…