I was trying to pass rows to dataloaders via DataBlock, but it seems that I haven’t get the complete clue of this, obviously passing an ndarray or the PILImage causes an exception too in some place because only collated types are allowed (but this tensor doesn’t have view).
well, I was not able to reproduce the loading of images from files… so I ended doing something like
if False:
def get_image(csv_row):
img = csv_row.reshape(28,28)
x = cm.gist_earth(img)*255
return Image.fromarray(np.uint8(x))
def explode(csv_name):
df = pd.read_csv(Path("digit-recognizer")/(csv_name+'.csv'), header='infer')
imagenes = df.iloc[:,1:].apply(lambda x: x.values, axis=1).values
labels = df.iloc[:,:1].apply(lambda x: x.values[0], axis=1)
for idx, l in enumerate(labels):
image_name = (str(l)+"_"+csv_name+'_'+''.join(random.choice(string.ascii_lowercase) for i in range(8))) + ".png"
p = Path(path)/image_name
im = get_image(imagenes[idx])
im.save(p)
explode("train")
It seems that is not possible do it like I wanted whic is to do it all on memory?? like:
- read the lines of the CSV, convert to images or nparrays and then pass them directly to dataloaders…
Is there a way to do it all on memory without save first the images? or what Im missing on my first implementation?