Train without Validation set

How to train a neural network without a validation set, because most of the dataset I have been seeing, most of them dont have the validation split. I want to implement hot dog classifier and tried to pass the directory, it returns an error “no /valid directory”. I know validation is neccessary, but since I’m beginner I want to train a model and try different datasets.


1 Like

I think learning the importance of how to validate models is the most crucial part any of any application. Here is a great blog post emphasizing that

Still after this if you want to continue without validation you may create a dummy validation folder and put a single image in it to run any model.


Thanks for the help!

1 Like

If you find it difficult to create a validation folder and move some random samples, pl use this script to move 20% of training set to validation set. (of course, you can modify the % of validation


Thanks @Renga this was needed!

1 Like

Another good source

tfms = tfms_from_model(arch, sz)
data =  ImageClassifierData.from_paths(PATH, bs, tfms=tfms, val_name='train', test_name='test')
from sklearn.model_selection import train_test_split
, val_idxs=train_test_split(range(len(y)),test_size=0.2, stratify=y)
data=ImageClassifierData.from_names_and_array(PATH, data.trn_ds.fnames, y, data.classes, 
                                         val_idxs=val_idxs, test_name='test',
                                         tfms=tfms, bs=bs)
len(data.trn_y), len(data.val_y)

this code could be of help.

1 Like

Thanks @lxkarthi!