How to do list comprehension in NumPy array

Deep Learning
1

I have a list of 3 channel image. I want to apply a function to every image. I start with list comprehension in python so: [fcn(img) for img in imgs] . However it is quite slow.

I think that NumPy can handle this better so I convert my list of image to 4D numpy array. However, I can’t find any source about doing this in NumPy. It seems have apply_along_axis or apply_over_axes or vectorize . However, I’m not succeed to make anything of these functions works.

Anyone can suggest me a way to handle this problem ? Thank you in advance

2

You are certainly on the right track. Numpy should be faster than raw Python list.

In terms of the implementation details, it really depends on the exact function you have in mind here. Could you please share with us what exact function you are trying to implement here?

1 Like
3

Thanks for your response. I was not thinking that it could depend on the inner function, but it might be the case.

I’m trying to extract the feature at resnet50 bottleneck (the layer before final layer). Because I want to infer on CPU so I try the OpenVINO framework of Intel to boosting the speed of inference. Everything is good now that it takes me 0.08s to run resnet50 so I can reach 12 FPS (quite fast in my opinion).

However, I am thinking about improving it. I want to run the inference on a bunch of images. So if it exist somehow to execute the function in parallel for every image, maybe I can improve the speed.

I was trying also the multiprocessing module of Python but not succeed. I check the CPU of my machine I find it uses 200% (I have 4 cores) so I didn’t use the full potential of my machine.

Here is my function (there is not fully details but you find the concept there):

    def extract_feature(imgs):
        in_frame = infer_en._prep_extract_feature(imgs)
        input_blob = infer_en.net_config['input_blob']
        pred = infer_en.exec_net.infer(inputs={input_blob: in_frame})[infer_en.output]
        pred = pred.flatten()
        return pred
4

I only have a very superficial understanding of the topics so please take my word with a grain of salt.

Numpy could dramatically speed up things if everything is written in Numpy, so that the clumsy Python operations that assumes everything to be an object with significant memory storage overhead now becomes efficient C operation that works directly on data stored efficiently in memory, with potential more speedup from SIMD parallelism.

This should not work if you are just using normal Python functions with Numpy data objects. So for your function, Numpy will probably not help much.

1 Like
5

Thanks @PegasusWithoutWinds . I think I will try more with the multiprocess. Your explanation of NumPy is very interesting :smiley:

6

Since fastai the core of the model is in PyTorch, I would try my luck with implementing everything with PyTorch operations and see if that helps.

I would also recommend using raw PyTorch methods rather than fastai methods when running inference. I know that this is really low-level stuff, but more convoluted OOP abstractions create more computational overhead, especially when they are not designed with speed in mind; even without the OOP abstractions, more layers of functions calling other functions are enough to make things a lot slower. Yes, there is an overhead to functional call as well.

1 Like
7

np.fromiter allows you to create arrays directly.

However, the speed is probably more a question about what you are doing in your “fcn”.

1 Like