I have built a model for detecting defects in a finished product. Here the approach is to localize the object and take multiple overlapping crops of the image and then classify each of these cropped images as ‘Good’ or ‘Defective’. I need to deploy the system to a linux machine locally.
As I need to get inferences for multiple image-crops simultaneously I want to use a GPU. I have the following questions in this regards:
1. How can I do learn.predict for multiple images simultaneously using GPU? 2. As I am only using the GPU for inference, what is the most cost effective solution in the form of a GPU box (or otherwise) on which I can install Fastai2 and deploy the model?