Hi
I’m kind of stuck on my work, and I’m looking for some advices.
I’m a image processing engineer. And I’m having a conventional image filter G, with a number of parameters w=(w1, w2, … wn).
So I’m going to have y_out = G(y_in, w), and the y_in and y_out is 2D gray level image.
And the G is not a blackbox, but it is quite complicated filter, so it takes a lot of try-and-errors, time, and effort to find a right or optimal parameters by hand-picking.
What I’m trying to do is let a deep learning model to learn to generate the parameter vector w based on the input image y_in and output image y_out. Then I think it can be considered as image regression problem. The input is images (y_in and y_out), and the output is the parameter vector w. Since the G is not the blackbox, I can reasonably randomly generate the w, so does y_out. So I can generate a lot of training/validation data.
Before I make a huge data and huge model, I just made a very small set with very few parameters like 10 parameters, w10=(w1,w2, … ,w10) for checking if the model is working okay. I generated 200 y_in/y_out/w10 pairs for training, and 30 pairs for validation.
I tried like giving both y_in and y_out to a deep learning model, for example resnet, and make the model to predict the parameter w, so I put sigmoid activation at the output neuron, and used MSE loss. It didn’t work well.
I tried few variations, like stacking y_in and y_out into channels, and give this as input to the network, or giving the difference y_out - y_in as input so the network can focus on the difference. Also tried using 2 networks for each images and concatenate them into fully connected layer. Tried different loss functions, tried different pre-trained model with different depth. But none of them gives good results. For example, the loss is going down, but the result is like most of the number are just zeros. Sometimes, the loss is not going down, or some cases, the output doesn’t seem to close to the desired value.
I’m still trying to find similar tasks so I can learn, but all I could find is image regression with very limited number of parameters like age prediction, head position prediction which is just one element or two element vector regression.
Is there any example that similar to the problem I’m trying to solve? Is there anything I need to look into? Is there anything I missed? Any advices will be very appreciated.
Thank you so much.