I’ve started a project to control a drone using various machine vision techniques by sending captured frames to be processed and decide how to control the drone. I have managed to hook up dlib with the drone and it is now following me around by my face.
The next step is integrating a depth perception model that I’ve been working on however I’m having a problem with the images coming in from the stream.
Problem
The UDP reads a stream and creates a np array but I can’t seem to get a fastai Image created properly from it.
n = np.load('test.npy')
# Convert from BGR
n = n[:,:,::-1]
p = PIL.Image.fromarray(n)
p.save('p1.jpg')
t = pil2tensor(p, dtype=np.uint8)
im = Image(t)
im.save('p2.jpg')
Any suggestions for possible things to do or if you interested in contributing, here is the Repository
By curious, so you are trying to guess the depth from RGB image ? I have image from with depth information (Intel Realsense camera), do you know some model that integrate the depth ? Thank you
Results aren’t too bad. Definitely could create some logic to use the results to move the drone to different areas.
Potential Next Steps:
Take the processed results and ask the drone to fly towards the piece with the highest 5 x 5 average CNN pool. i.e. the direction which is least explored whilst making sure that the average of the resulting tensor is greater than a certain amount (Flying into walls is bad).
Map out a room (or do as good of a job as possible), this could be possible if the drone knows how much it is moving around but I’m not sure the results are good enough.