Been working through the notebook for lesson 2. Ran into some error messages in regards to memory when running lines that call get_data. As suggested above I switched to get_batches. Well when you try to run trn_data.shape I get the following error message: AttributeError: 'DirectoryIterator' object has no attribute 'shape' So I checked out the get_data function and it uses the get_batches and then the return statement appears to be concatenating all of the batches. Thats the line that I believe is causing my memory issues. Below you will see the code I used to sort of play and isolate what I believe is my issue. Has anybody else had that error or might have some suggestions as how to over come it? Thank you.
PS I'm running this on my own machine NVIDIA GTX 1080 8gigs
val_data = get_batches(path+'valid', shuffle=False, batch_size=1, class_mode=None, target_size=(224,224)) #this runs fine.
val_data = np.concatenate([val_data_a.next() for i in range(val_data_a.nb_sample)]) #this runs fine small amount of images to process
Found 2000 images belonging to 2 classes.
trn_data = get_batches(path+'train', shuffle=False, batch_size=1, class_mode=None, target_size=(224,224)) # this runs fine
trn_data = np.concatenate([trn_data_a.next() for i in range(trn_data_a.nb_sample)]) # this blows out my memory
Found 23000 images belonging to 2 classes.
AttributeError Traceback (most recent call last)
----> 1 trn_data.shape
AttributeError: 'DirectoryIterator' object has no attribute 'shape'