Using image data with tabular data as input

I am fairly new to deep learning and fastai, but did 2018 part 1.

Let’s say I have images and for each image I also have additional information like categorical variables, real values etc. I would be interested at predicting a binary output for each image (also using the structured data associated for each of those images).

Then once everything is trained look at the embeddings of my categorical variables and maybe use that for other tasks.

Any easy way to do that in fastai (using images + structured data combined)?

Thank you!


You have to write a custom data block, but it’s very doable. Wayde Gilliam (not sure his handle here) has a great example of doing so for tabular+text:

along with a corresponding blog post:

That should help get you started.

Hope that helps.


Thank you, will definitely read that!