Split data using fit_generator


(Gohar Irfan Chaudhry) #1

Is there a way to split input data into validation and training using fit_generator like in the normal fit method? By specifying what fraction of data we want to use for validation?


(Rob H) #2

Hi Gohar, I don’t see anything in the Keras documentation for that. You probably already moved on past this…


#3

here is an example.

https://www.kaggle.com/dromosys/dogs-vs-cats-keras/

split the training validation data
train_images, validation_images = train_test_split(images, test_size=0.4)

and sample size is set by
images = train_dogs[:200] + train_cats[:200]


(Richard Horton) #4

This functionality has been added to Keras:

train_datagen = ImageDataGenerator(rescale=1./255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    validation_split=0.2)

train_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary',
    subset='training')

validation_generator = train_datagen.flow_from_directory(
    train_data_dir,
    target_size=(img_width, img_height),
    batch_size=batch_size,
    class_mode='binary'
    subset='validation')

model.fit_generator(
    train_generator,
    steps_per_epoch = train_generator.samples // batch_size,
    validation_data = validation_generator, 
    validation_steps = validation_generator.samples // batch_size,
    epochs = nb_epochs)

https://keras.io/preprocessing/image/