Hello,
I am currently working with thermal images from solar PV modules. The idea is detect issues in PV plants with drone aerial imaging.
Currently, we work with a drone and a thermal camera that takes a video (300x300 I know, is bad, but thermal cameras are not as good as regular cameras). Then, we build a huge image (100MP or more) that represent the PV plant.
My DL idea is to cut this huge image in 256x256 pieces(or more) and segment the PV panels, what I need as segmentation is the four corners of the PV modules, to be able to compute information of the PV module.
Left images is the thermal camera image, Right images is the location of the vertex of modules.
How can I create this data automatically, maybe training in synthetic images of lines and rectangles?
Maybe is better to do masks of rectangles, to address the issue of deformed straight lines?
Thanks for any insight that can save me to segment manually thousand of images…
Anyway, I think the first hurdle will be to get the ground truth of where the borders of the panels are located. Possible actions:
a) find an existing dataset that is more or less close to yours - might be difficult
b) simulate the panels i 3D. it design one panel use it to generate alle panels . Know you can take snapshot from different angles, introduce fog , light, dirt etc.
teh last option is an investment but i have seen it applied succesfully to plant/weeds.
To start out, I’d recommend training a UNet to learn pixel-wise probabilities of each pixel being a corner, with a simple Euclidean softmax loss function. The context for a given pixel output of the net should be wide enough to include an entire PV panel.
UNets are generally useful for segmentation problems. I’ve done research on image segmentation with UNets, and they work very well combined with Euclidean loss and other loss functions for boundary detection and neighbor-affinity (i.e., probability that the next pixel along some axis is part of the same blob).
Shoot me an email if you want specialized help on this. I have a few other ideas to help you structure this problem – how to preprocess your data, structuring your labels, loss functions to use, etc.
Your are the best, yes I was thinking on doing resnet34+DyanmicUnet as starting point, but for that I need labels…
I am playing with sketchup to construct some solar panels.
If that architecture is difficult to use, i bet you could get pretty good results with a vanilla UNet. And without too much data, with the right data augmentation and hyperparameters.
I have as many real images as I want, I have 4 hours of video (3 complete centrals).
I am trying to find a way to create segmasks from the sketchup.
To give you some more info about the problem, I need to compute for each solar panel (from the values of the temperatures) a function that will determine if the module is malfunctioning.
So I need to be able to identify each solar panel individually, to be able to compute this function.
My function uses now the 4 points of the vertex, to reference the area inside, and compute the function.
Irght now, this is done by hand, image per image I have to click the 4 points, at the defectuos panel to compute my funciton.
For a big central, with hundred of thousand of modules, this is not possible.
If you don’t have the resources to label, you could try pseudo labelling or semi-supervised training.
e.g. build a binary classifier (labelling much easier) to narrow down candidate images
e.g. train on 1000 image segmasks to predict 10k more, and use those to ease hand labelling of the 10k, rinse and repeat.
e.g. You could just use the noisy (predicted) labels from a dataset as a new train dataset.
I haven’t worked more on the problem, as we are changing approach. I worked a bit with hand-made segmasks in this repo: https://github.com/tcapelle/solar_seg
