Image normalization consumes too much RAM

cap_rogers · September 23, 2019, 11:33am

I am doing an image recognition task, I converted the input images to a list of array of pixels. When i try to normalize it , I am getting a Memory Error

kushaj · September 23, 2019, 6:22pm

Does this work?

X_M = np.array(X_m)
x_M /= 255

cap_rogers · September 24, 2019, 7:15pm

That works fine but I always get a memory error when I try to split the data

kushaj · October 15, 2019, 7:39pm

Maybe converting numpy array back to python list might help.

SakvaUA · October 15, 2019, 10:24pm

When you divide image by 255 numpy converts it from uint8 to float32, which takes 4x as much memory. That’s why it is more memory efficient to store images in raw uint8 format and normalize them one at a time (in Dataset)