Almost finished video 7 now. The course was super good at showing us how to use what fastai provide to solve well studied problems like image classification, image segmentation etc, but it could also be interesting in the future to help us think how to solve novel problems by using the building blocks fastai provide.
I have several ideas on how to apply those techniques at work, but a lot of those ideas would have to combine various kinds of data (images + text + tabular). Right now if my problem is not in those well defined bins that fastai do very very well, there’s less information out there. Where do I start, how should I think while designing my neural network architecture etc etc.
For example I would like to predict the likelihood of food safety failure in a restaurant from pictures taken by inspectors in the past. Inspectors respond to questions (text), the question can pass or fail and then they associate pictures to those. So I have pictures, those pictures are in the context of a question (text) and the question can either have passed or failed and be in the context of a store (tabular).
I can clearly see that I could use pre-trained image model for the images and combined their encoded representation somehow, pre-trained language model for the question context and use the encoded representation there too, use embeddings for my stores categorical information and combine all this together…
I am slowly reading fastai code to try to figure out how I could implement something like that but I feel like it would be really really nice to have a tutorial from end to end on how to implement a novel scenario in fast ai. Implement the ItemBase, ItemList, databunch, make a new model specifically for your scenario, using pre-trained models etc.
Looking forward to part 2!!